CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-pybullet

Official Python Interface for the Bullet Physics SDK specialized for Robotics Simulation and Reinforcement Learning

Pending
Overview
Eval results
Files

vr-input-handling.mddocs/

VR and Input Handling

Virtual reality support, keyboard and mouse input handling for interactive simulations, teleoperation, and user interfaces. Essential for immersive simulation experiences and human-in-the-loop applications.

Capabilities

VR Support

def getVREvents(deviceTypeFilter=None, physicsClientId=0):
    """
    Get VR controller events and state.
    
    Args:
        deviceTypeFilter (int, optional): Filter by device type
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    
    Returns:
        list: VR events with controller positions, orientations, and button states
    """

def setVRCameraState(cameraDistance, yaw, pitch, cameraTargetPosition, physicsClientId=0):
    """
    Set VR camera properties.
    
    Args:
        cameraDistance (float): Distance from target
        yaw (float): Yaw angle in degrees
        pitch (float): Pitch angle in degrees
        cameraTargetPosition (list): Target position [x, y, z]
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    """

Input Events

def getKeyboardEvents(physicsClientId=0):
    """
    Get keyboard input events.
    
    Args:
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    
    Returns:
        dict: Dictionary of keyboard events with key codes and states
              Format: {keycode: keystate} where keystate includes:
              - wasTriggered: whether key was pressed this frame
              - isPressed: whether key is currently held down
    """

def getMouseEvents(physicsClientId=0):
    """
    Get mouse input events.
    
    Args:
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    
    Returns:
        list: List of mouse events containing:
              - eventType: type of mouse event (MOUSE_MOVE, MOUSE_BUTTON, etc.)
              - mousePosX, mousePosY: mouse position
              - buttonIndex: mouse button index
              - buttonState: button state (pressed/released)
    """

Utility Functions

def getAPIVersion():
    """
    Get PyBullet API version.
    
    Returns:
        int: API version number
    """

def isNumpyEnabled():
    """
    Check if NumPy support is enabled.
    
    Returns:
        bool: True if NumPy is available for array operations
    """

def setAdditionalSearchPath(path):
    """
    Add additional search path for loading URDF and other data files.
    
    Args:
        path (str): Directory path to add to search paths
    """

def setTimeOut(timeOutInSeconds, physicsClientId=0):
    """
    Set timeout for physics simulation operations.
    
    Args:
        timeOutInSeconds (float): Timeout duration in seconds
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    """

Plugin System

def loadPlugin(pluginPath, postFix="", physicsClientId=0):
    """
    Load physics engine plugin.
    
    Args:
        pluginPath (str): Path to plugin library
        postFix (str, optional): Plugin name postfix. Defaults to "".
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    
    Returns:
        int: Plugin unique ID
    """

def unloadPlugin(pluginUniqueId, physicsClientId=0):
    """
    Unload physics engine plugin.
    
    Args:
        pluginUniqueId (int): Plugin unique ID from loadPlugin
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    """

def executePluginCommand(pluginUniqueId, textArgument="", intArgs=None, floatArgs=None, physicsClientId=0):
    """
    Execute command in loaded plugin.
    
    Args:
        pluginUniqueId (int): Plugin unique ID
        textArgument (str, optional): Text argument for plugin. Defaults to "".
        intArgs (list, optional): Integer arguments for plugin
        floatArgs (list, optional): Float arguments for plugin
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    
    Returns:
        int: Plugin execution result
    """

Advanced Functions

def vhacd(fileName, fileNameOut, alpha=0.04, resolution=100000, depth=20, concavity=0.025, planeDownsampling=4, convexhullDownsampling=4, pca=0, mode=0, maxNumVerticesPerCH=64, minVolumePerCH=0.0001):
    """
    Perform volumetric hierarchical approximate convex decomposition (V-HACD).
    
    Args:
        fileName (str): Input mesh file path
        fileNameOut (str): Output file path for decomposed mesh
        alpha (float, optional): Controls surface vs volume sampling. Defaults to 0.04.
        resolution (int, optional): Voxel resolution. Defaults to 100000.
        depth (int, optional): Maximum number of clipping planes. Defaults to 20.
        concavity (float, optional): Maximum allowed concavity. Defaults to 0.025.
        planeDownsampling (int, optional): Plane downsampling rate. Defaults to 4.
        convexhullDownsampling (int, optional): Convex hull downsampling. Defaults to 4.
        pca (int, optional): Enable/disable PCA. Defaults to 0.
        mode (int, optional): Decomposition mode. Defaults to 0.
        maxNumVerticesPerCH (int, optional): Max vertices per convex hull. Defaults to 64.
        minVolumePerCH (float, optional): Min volume per convex hull. Defaults to 0.0001.
    
    Returns:
        list: List of decomposed convex hulls
    """

def submitProfileTiming(name, physicsClientId=0):
    """
    Submit profile timing information.
    
    Args:
        name (str): Profile timing name/label
        physicsClientId (int, optional): Physics client ID. Defaults to 0.
    """

Usage Examples

Keyboard and Mouse Input

import pybullet as p
import time

p.connect(p.GUI)
p.loadURDF("plane.urdf")
cube_id = p.loadURDF("cube_small.urdf", [0, 0, 1])

print("Use WASD keys to move the cube, mouse to look around")
print("Press ESC to exit")

cube_pos = [0, 0, 1]

while True:
    # Get keyboard events
    keys = p.getKeyboardEvents()
    
    # Handle key presses
    if ord('w') in keys and keys[ord('w')] & p.KEY_IS_DOWN:
        cube_pos[1] += 0.01
    if ord('s') in keys and keys[ord('s')] & p.KEY_IS_DOWN:
        cube_pos[1] -= 0.01
    if ord('a') in keys and keys[ord('a')] & p.KEY_IS_DOWN:
        cube_pos[0] -= 0.01
    if ord('d') in keys and keys[ord('d')] & p.KEY_IS_DOWN:
        cube_pos[0] += 0.01
    
    # Check for ESC key to exit
    if p.B3G_ESCAPE in keys and keys[p.B3G_ESCAPE] & p.KEY_WAS_TRIGGERED:
        break
    
    # Update cube position
    p.resetBasePositionAndOrientation(cube_id, cube_pos, [0, 0, 0, 1])
    
    # Get mouse events
    mouse_events = p.getMouseEvents()
    for event in mouse_events:
        if event[0] == p.MOUSE_MOVE_EVENT:
            print(f"Mouse moved to: {event[1]}, {event[2]}")
    
    p.stepSimulation()
    time.sleep(1./60.)

p.disconnect()

VR Support

import pybullet as p
import time

# Connect with VR support (requires VR headset)
try:
    p.connect(p.SHARED_MEMORY)  # or p.GUI with VR enabled
    
    # Load scene
    p.loadURDF("plane.urdf")
    robot_id = p.loadURDF("r2d2.urdf", [0, 0, 1])
    
    # Set up VR camera
    p.setVRCameraState(
        cameraDistance=2.0,
        yaw=0,
        pitch=-30,
        cameraTargetPosition=[0, 0, 1]
    )
    
    print("VR mode active - use VR controllers to interact")
    
    for i in range(1000):
        # Get VR controller events
        vr_events = p.getVREvents()
        
        for event in vr_events:
            # Process VR controller input
            # event contains controller position, orientation, button states
            controller_pos = event[1]  # Controller position
            controller_orn = event[2]  # Controller orientation
            buttons = event[6]         # Button states
            
            # Example: Use trigger to apply force
            if buttons[33] & p.VR_BUTTON_IS_DOWN:  # Trigger pressed
                p.applyExternalForce(
                    robot_id, -1, [0, 0, 5], controller_pos, p.WORLD_FRAME
                )
        
        p.stepSimulation()
        time.sleep(1./90.)  # VR typically runs at 90Hz
        
except Exception as e:
    print(f"VR not available: {e}")
    # Fall back to regular GUI mode
    p.connect(p.GUI)

Plugin System Usage

import pybullet as p

p.connect(p.GUI)

# Load a custom physics plugin
try:
    plugin_id = p.loadPlugin("custom_physics_plugin.so")
    print(f"Loaded plugin with ID: {plugin_id}")
    
    # Execute plugin command
    result = p.executePluginCommand(
        plugin_id,
        textArgument="initialize",
        intArgs=[1, 2, 3],
        floatArgs=[0.1, 0.2, 0.3]
    )
    print(f"Plugin command result: {result}")
    
    # Use plugin during simulation...
    
    # Unload plugin when done
    p.unloadPlugin(plugin_id)
    print("Plugin unloaded")
    
except Exception as e:
    print(f"Plugin loading failed: {e}")

Utility Functions

import pybullet as p
import pybullet_data

# Check API version and NumPy support
print(f"PyBullet API version: {p.getAPIVersion()}")
print(f"NumPy enabled: {p.isNumpyEnabled()}")

# Set up data search paths
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setAdditionalSearchPath("./custom_models/")

# Connect and configure timeout
p.connect(p.GUI)
p.setTimeOut(10.0)  # 10 second timeout for operations

# Load objects using search paths
plane_id = p.loadURDF("plane.urdf")  # Found via search path
robot_id = p.loadURDF("r2d2.urdf")   # Found via search path

print("Simulation ready with configured search paths and timeout")

Input Constants

# Keyboard key constants
p.KEY_IS_DOWN        # Key is currently pressed
p.KEY_WAS_TRIGGERED  # Key was just pressed this frame
p.KEY_WAS_RELEASED   # Key was just released this frame

# Special keys
p.B3G_ESCAPE        # Escape key
p.B3G_RETURN        # Enter key
p.B3G_SPACE         # Space bar
p.B3G_LEFT_ARROW    # Left arrow key
p.B3G_RIGHT_ARROW   # Right arrow key
p.B3G_UP_ARROW      # Up arrow key
p.B3G_DOWN_ARROW    # Down arrow key

# Mouse events
p.MOUSE_MOVE_EVENT   # Mouse movement
p.MOUSE_BUTTON_EVENT # Mouse button press/release

# VR button constants  
p.VR_BUTTON_IS_DOWN     # VR button pressed
p.VR_BUTTON_WAS_TRIGGERED # VR button just pressed
p.VR_BUTTON_WAS_RELEASED  # VR button just released

Best Practices

  1. Event Processing - Check input events every frame to avoid missing user interactions
  2. Key State Checking - Use appropriate key state flags (IS_DOWN vs WAS_TRIGGERED) for different behaviors
  3. VR Frame Rate - Maintain 90Hz for VR applications to prevent motion sickness
  4. Plugin Safety - Always unload plugins properly to prevent memory leaks
  5. Search Path Management - Set search paths early in your application initialization
  6. Timeout Configuration - Set reasonable timeouts for network-based physics clients
  7. Input Validation - Validate plugin commands and file paths before execution

Install with Tessl CLI

npx tessl i tessl/pypi-pybullet

docs

collision-detection.md

connection-simulation.md

index.md

inverse-kinematics-dynamics.md

joint-motor-control.md

mathematical-utilities.md

object-loading.md

physics-configuration.md

rendering-visualization.md

state-management-logging.md

vr-input-handling.md

tile.json