Setting Up the Agipix Simulation Environment

Prerequisites for host PC

Ubuntu 20.04/22.04 Operating System
NVIDIA GPU (RTX 2070 or higher)
NVIDIA GPU Driver (recommended version 525.85)

ROS2 Humble

Follow the latest instructions at the official page

tmux

sudo apt install tmux
cd && git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm

if you are new to tmux or not already familiar wit tmux

wget https://raw.githubusercontent.com/SasaKuruppuarachchi/SasaKuruppuarachchi/main/.tmux.conf -P ~/

go to .tmux.conf and edit line 12 “set -g prefix `”. This charactor “`” depends on your keyboard layout.

IsaacSIM Installation

This documentation is tested for the current stable version 5.1.0. This will be updated as new stable release published

Quick install instructions: Official Documentation

mkdir isaac-sim
cd isaac-sim/
wget https://download.isaacsim.omniverse.nvidia.com/isaac-sim-standalone-5.1.0-linux-x86_64.zip
unzip "isaac-sim-standalone-5.1.0-linux-x86_64.zip" -d ~/isaacsim
rm isaac-sim-standalone-5.1.0-linux-x86_64.zip # optionally clean install artifacts
cd ~/isaacsim
./post_install.sh

Setup environemnt vars

cat <<'EOF' >> ~/.bashrc

# ---------------------------
# ISAAC SIM SETUP
# ---------------------------
# Isaac Sim root directory
export ISAACSIM_PATH="${HOME}/isaacsim"
# Isaac Sim python executable
export ISAACSIM_PYTHON="${ISAACSIM_PATH}/python.sh"
# Isaac Sim app
export ISAACSIM="${ISAACSIM_PATH}/isaac-sim.sh"

# Define an auxiliary function to launch Isaac Sim or run scripts with Isaac Sim's python
# This is done to avoid conflicts between ROS 2 and Isaac Sim's Python environment
isaac_run() {

    # ------------------
    # === VALIDATION ===
    # ------------------
    if [ ! -x "$ISAACSIM_PYTHON" ]; then
        echo "❌ IsaacSim python.sh not found at: $ISAACSIM_PYTHON"
        return 1
    fi
    if [ ! -x "$ISAACSIM" ]; then
        echo "❌ IsaacSim launcher not found at: $ISAACSIM"
        return 1
    fi

    # -------------------------
    # === CLEAN ENVIRONMENT ===
    # -------------------------
    # Unset ROS 2 environment variables to avoid conflicts with Isaac's Python 3.11
    unset ROS_VERSION ROS_PYTHON_VERSION ROS_DISTRO AMENT_PREFIX_PATH COLCON_PREFIX_PATH PYTHONPATH CMAKE_PREFIX_PATH

    # Remove ROS 2 paths from LD_LIBRARY_PATH if present
    local ros_paths=("/opt/ros/humble" "/opt/ros/jazzy" "/opt/ros/iron")
    for ros_path in "${ros_paths[@]}"; do
        export LD_LIBRARY_PATH=$(echo "$LD_LIBRARY_PATH" | tr ':' '\n' | grep -v "^${ros_path}" | paste -sd':' -)
    done

    # -----------------------------
    # === UBUNTU VERSION CHECK ===
    # -----------------------------

    if [ -f /etc/os-release ]; then
        UBUNTU_VERSION=$(grep "^VERSION_ID=" /etc/os-release | cut -d'"' -f2)
    fi

    # If Ubuntu 24.04 -> use the Isaac Sim internal ROS2 Jazzy (ROS2 Jazzy bridge)
    if [[ "$UBUNTU_VERSION" == "24.04" ]]; then
        export ROS_DISTRO=jazzy
        export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
        export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${ISAACSIM_PATH}/exts/isaacsim.ros2.bridge/jazzy/lib"
        echo "🧩 Detected Ubuntu 24.04 -> Using ROS_DISTRO=jazzy"
    # If Ubuntu 22.04 -> use the Isaac Sim internal ROS2 Humble (ROS2 Humble bridge)
    else
        export ROS_DISTRO=humble
        export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
        export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${ISAACSIM_PATH}/exts/isaacsim.ros2.bridge/humble/lib"
        echo "🧩 Detected Ubuntu ${UBUNTU_VERSION:-unknown} -> Using ROS_DISTRO=humble"
    fi

    # ---------------------
    # === RUN ISAAC SIM ===
    # ---------------------
    if [ $# -eq 0 ]; then
        # No args → Launch full Isaac Sim GUI
        echo "🧠 Launching Isaac Sim GUI..."
        "${ISAACSIM}"

    elif [[ "$1" == --* ]]; then
        # Arguments start with "--" → pass them to Isaac Sim executable
        echo "⚙️  Launching Isaac Sim with options: $*"
        "${ISAACSIM}" "$@"

    elif [ -f "$1" ]; then
        # First argument is a Python file → run with Isaac Sim's Python
        local SCRIPT_PATH="$1"
        shift
        echo "🚀 Running Python script with Isaac Sim: $SCRIPT_PATH"
        "${ISAACSIM_PYTHON}" "$SCRIPT_PATH" "$@"

    else
        # Unrecognized input
        echo "❌ Unknown argument or file not found: '$1'"
        echo "Usage:"
        echo "  isaac_run                 → launch GUI"
        echo "  isaac_run my_script.py    → run script with IsaacSim Python"
        echo "  isaac_run --headless ...  → launch IsaacSim with CLI flags"
        return 1
    fi
}
alias agisim="cd ~/workspace/raicam-ros/src/isaac_ros_common/ && ./run_sim.sh"
export ROS_DOMAIN_ID=3
EOF
source ~/.bashrc

Running IsaacSIM

Nucleus Server

Depreciated

Download and install Omniverse Launcher and install Nucleus Server Go to the library and install ISAACSIM.

Enable ROS2 extension

Open IsaacSIM > Window > Extension > ROS2 bridge – enable (Autoload)

Installing Qgroundcontrol (for PX4 based implementations)

Follow the latest instructions at the official page

sudo usermod -aG dialout "$(id -un)"

(Optional) Disable ModemManager On some Ubuntu-based systems, ModemManager can claim serial ports that QGC needs. If you don’t use it elsewhere, mask or remove it.

# preferred: stop and mask the service
sudo systemctl mask --now ModemManager.service

# or, if you’d rather remove the package
sudo apt remove --purge modemmanager

On the command prompt, enter:

sudo apt install gstreamer1.0-plugins-bad gstreamer1.0-libav gstreamer1.0-gl -y
sudo apt install libfuse2 -y
sudo apt install libxcb-xinerama0 libxkbcommon-x11-0 libxcb-cursor-dev -y

To install QGroundControl:

Download QGroundControl-x86_64.AppImage.
Make the AppImage executable

chmod +x QGroundControl-x86_64.AppImage
./QGroundControl-x86_64.AppImage

PX4 firmware

# Linux packages
sudo apt install git make cmake python3-pip

# Python packages
pip install kconfiglib jinja2 empy jsonschema pyros-genmsg packaging toml numpy future

cd $HOME
git clone https://github.com/PX4/PX4-Autopilot.git -b v1.16.1 --recursive
cd PX4-Autopilot
make px4_sitl_default none

if PX4 launched properly you can exit out of it. For further information visit here

(Optional) If an `externally-managed-environment` Use an venv

sudo apt install python3-venv python3-pip python3-full
cd PX4-Autopilot
# setup and activate venv
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install empy jinja2 numpy toml pyyaml packaging
# install and fix python deps
pip install kconfiglib jinja2 empy jsonschema pyros-genmsg packaging toml numpy future
pip uninstall empy
pip install empy==3.3.4
# Build Px4 
make clean
make px4_sitl_default none

To synchronise the time stamps with the simulator we need to set the parameter UXRCE_DDS_SYNCT to false. To do that,

Open the rcS file at /PX4-Autopilot/ROMFS/px4fmu_common/init.d-posix/rcS

Paste the following line before ´#Autostart ID´

param set-default UXRCE_DDS_SYNCT 0
param set-default UXRCE_DDS_DOM_ID 3

Rebuild the firmware

cd $HOME
cd PX4-Autopilot
make px4_sitl_default none

Make sure to set the parameter use_sim_time (of each ROS2 node) to true

Pegusus simulaotor

Cloning into the repo and the assets

cd $HOME
git clone https://github.com/SasaKuruppuarachchi/PegasusSimulator.git
cd PegasusSimulator/extensions/pegasus.simulator/pegasus/simulator/assets/Worlds
git clone https://github.com/SasaKuruppuarachchi/isaac_envs.git

Installing the extension as a library

cd ~/isaacsim
./python.sh -m pip install --editable  /home/sasa/PegasusSimulator/extensions/pegasus.simulator

[!NOTE] For the first time, launching Isaac Sim takes a very long time. Isaac Sim must be fully launched to spawn the robot.

Enabling Pegasus Simulator in Isaacsim (First Operation)

Launch isaac_run application.
Open the Window->extensions on the top menubar inside Isaac Sim.
On the Extensions manager menu, we can enable or disable extensions. By pressing the settings button, we can add a path to the Pegasus-Simulator repository.
The path inserted should be the path to the repository followed by /extensions.
After adding the path to the extension, we can enable the Pegasus Simulator extension on the third-party tab.

When enabling the extension for the first time, the python requirements should be install automatically for the build in ISAACSIM_PYTHON , and after a few seconds, the Pegasus widget GUI should pop-up.

The Pegasus Simulator window should appear docked to the bottom-right section of the screen.

Post setups

Download assets

Download assets from Official page and same them in ~/isaacsim/assets/
then mkdir ~/isaacsim/assets/Isaac/5.1/NVIDIA

Open ~/isaacsim/exts/isaacsim.asset.browser/config/extension.toml and add the asset paths at

  [settings]
  # Root folder URLs to list and monitor.
  exts."isaacsim.asset.browser".folders = [
      # asset paths here
  ]

Open ~isaacsim/exts/isaacsim.storage.native/config/extension.toml and replace persistent.isaac.asset_root.default with persistent.isaac.asset_root.default = "/home/sasa/isaacsim/assets/Isaac/5.1"

Add Lidar definitions

To replicate Mid360 lidar in the Agipix we create this custom Lidar model in Isaacsim.

Copy Mid360 usda file cp ~/PegasusSimulator/assets/lidar/Mid_360.usda ~/isaacsim/assets/Isaac/5.1/Isaac/Sensors/NVIDIA/Mid_360.usda
Open ~/isaacsim/exts/isaacsim.sensors.rtx/isaacsim/sensors/rtx/impl/supported_lidar_configs.py and add "/Isaac/Sensors/NVIDIA/Mid_360.usda": set(), under the SUPPORTED_LIDAR_CONFIGS

Running the Simulation

Open a new terminal to launch the simulator

cd $ISAAC_ROS_DEV_DIR/src/isaac_ros_common/ && ./run_sim.sh
# press ENTER after moving to the right pane to launch IsaacSim

In a new terminal run the docker container
```
runagi
```
Controlling the drone

Follow the instructions in the top right pane of docker tmux window

Opening the UI
1. Open up a Foxglow account and bavigate to the dashboard
2. Open connection with “ws://localhost:8765”

Now we are ready for autonomy setup. See Instructions to continue.

← Back to Home View on GitHub