Quick Start¶

This document describes the current JSON environment workflow for EAI Simulator. The project is currently based on Isaac Sim 5.1, Isaac Lab 2.x and the env_isaaclab conda environment.

Prerequisites¶

Isaac Sim 5.1 and Isaac Lab 2.x are installed.
Use the env_isaaclab conda environment including Isaac Lab dependencies.
Runs on both CPU and CUDA GPU.

Install¶

Clone the repository and enter the repository root directory and run:

source "$(conda info --base)/etc/profile.d/conda.sh"
conda activate env_isaaclab

python -m pip install --no-deps -e source/EAI
python -m pip install --no-deps -e source/EAI_assets
python -m pip install --no-deps -e source/EAI_hmrs

Also available:

./tools/setup/install_packages.sh

Environment files¶

There is one JSON for each bootable environment:

source/EAI_hmrs/EAI_hmrs/envs/<env_name>.json

View available environments:

find source/EAI_hmrs/EAI_hmrs/envs -maxdepth 1 -name '*.json' -printf '%f\n' | sort

Select running mode¶

Command	Interface	Applicable scenarios	Pose source
`python simulator.py`	Env DIY startup menu (select `1`/`2`/`3` after launch)	Choose the lightweight window, terminal setup, or 3D editor	Depends on the selected mode
`python simulator.py --diy-3d`	Isaac Sim right side Env DIY 3D plug-in	Requires real collision surface, height and 3D drag	Real `spawn_pose`
`python simulator.py --env=<name>`	Go directly to the official simulation	Start a saved JSON environment	`spawn_pose` in JSON or the default location

When --env is omitted, select 3 from the startup menu to open the 3D editor; --diy-3d remains available as a direct entry point. After completing the pre-run editing, click Run in the plug-in, and the program will switch from the preview stage to the formal environment in the same Isaac Sim/Kit process, and Isaac Sim will not be closed and reopened.

Current status: --diy-3d is still in the continuous optimization stage. It is recommended for development, asset verification and process joint debugging; plug-in layout, asset catalog and some controller interfaces may be adjusted with the version. Please keep the exported selection JSON for formal experiments.

Start an existing environment¶

python simulator.py --env robo

This is the quick start command recommended by the project. --env robo will read:

source/EAI_hmrs/EAI_hmrs/envs/robo.json

The robo environment loads supported heterogeneous robots in a flat scene and enables keyboard control for each object. Run registry-driven human assets separately with python -u tools/human_assets/run_demo.py; see Human Asset Development for the complete capability contract.

Env DIY¶

Enter Env DIY without passing --env:

python simulator.py --num_envs=1 --device=cuda:0

Available options:

-Visualization window: Press Scenes Ўъ Robots Ўъ Payloads Ўъ Tools to drag the scene, host robot, robot arm/sensor and control tool.

Terminal Quick: Select Scene, Host Robot, Manipulators, Sensors and Tools in the same order.

The saved environment is written directly to source/EAI_hmrs/EAI_hmrs/envs/, and then started again by saving the name.

Use Isaac Sim 3D mode when you need to edit according to a true 3D position in the simulation world:

python simulator.py --diy-3d --device=cuda:0

This mode opens the Isaac Sim 3D editor before the official simulation is run, and is used to save the real robot position. For detailed 3D editing, asset preparation and failure recovery processes, see Environment Description.

--diy-3d launches the Isaac Sim AppLauncher only once. After clicking Run, the program completes the environment transition in the same Kit process. For asset downloads, manipulator control, and troubleshooting, see the Environment Guide and Manipulator Control.

External keyboard control¶

The repository retains the minimal keyboard environment source/EAI_hmrs/EAI_hmrs/envs/keyboard.json. Start first:

python simulator.py --env=keyboard --device=cuda:0

This environment builds carter_1 and enables /carter_1/cmd_vel. From another terminal run:

source /opt/ros/humble/setup.bash
python3 algorithm/keyboard/keyboard.py --robot carter_1

Other temporary environments can still add the keyboard tool through Env DIY, and the instance names are generated by type and order.

The keyboard environment of MuSHR Nano v2 is mushr_v2_keyboard, using MuSHR’s own Ackermann controller and 0.15 m/s pure angular velocity minimum forward speed:

python simulator.py --env=mushr_v2_keyboard --device=cuda:0
source /opt/ros/humble/setup.bash
/usr/bin/python3 algorithm/keyboard/keyboard.py --robot mushr_v2_1 --linear-speed 0.5 --angular-speed 0.8

W/S controls linear.x, A/D controls linear.y, R/F controls aerial ascent/descent through linear.z, and C/V controls angular.z; MuSHR does not use linear.y and cannot rotate in place. Use --vertical-speed to configure the aerial vertical speed separately.

All objects in the robo environment also have the keyboard tool configured. After the emulation is started, you can run the unified keyboard publisher in another terminal:

/usr/bin/python3 algorithm/keyboard/keyboard.py

Nav2 Example¶

The repository contains Factory + Carter’s Nav2 JSON example:

# Terminal 1: Isaac Sim GUI
conda activate env_isaaclab
python simulator.py --env=nav2 --device=cuda:0

# Terminal 2: Nav2 and RViz
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
ros2 launch algorithm/nav2/nav2.launch.py \
  robot_name:=carter_1 robot_type:=Carter scene:=factory rviz:=true

# Terminal 3: Send target
source /opt/ros/humble/setup.bash
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
/usr/bin/python3 algorithm/nav2/send_goal.py --x -5.0 --y -8.0

This command exits with status 0 only when Nav2 returns STATUS_SUCCEEDED; an unavailable server, rejected goal, canceled goal, aborted goal, or timeout returns a nonzero status. Ctrl+C in the send_goal.py terminal stops only the goal client. To stop Nav2 and Isaac Sim started by the one-command launcher, press Ctrl+C in the run_nav2.sh terminal and let cleanup finish.

Fire Rescue Demo¶

The pure robot fire rescue experiment uses the same Simulator interface and EAI-Factory-v0.json:

python -m demo.fire_rescue.main \
  --env=EAI-Factory-v0 \
  --device=cuda:0 \
  --trials=1 \
  --trial-hazard-ids=1 \
  --auto-fire-delay=0 \
  --emos-llm-preset=zhipu-glm4-flash

After non-headless is started, the terminal will prompt the monitoring page:

http://127.0.0.1:8767/

For the complete structure and algorithm calling method, see demo/fire_rescue/README.md.

Verification¶

Check the CLI first, without starting Isaac Sim:

python simulator.py --help
python -m demo.fire_rescue.main --help

Start another JSON environment for real verification:

python simulator.py --env robo

FAQ¶

`ModuleNotFoundError`¶

Re-execute the three editable installation commands and confirm that env_isaaclab is activated in the current terminal.

Environment JSON not found¶

Verify that the file exists and --env does not contain .json:

ls source/EAI_hmrs/EAI_hmrs/envs/EAI-Factory-v0.json

CUDA out of memory¶

Keep --num_envs=1 to close other GPU-hogging processes.

ROS2 topic is not published¶

Confirm that the robot contains a navigation_io or keyboard attachment in the JSON, and check the Isaac Sim ROS2 Bridge and ROS Humble environment variables.