Skip to content

Add dVRK needle-pass environment and OpenXR teleoperation#6527

Draft
chrisvoncsefalvay wants to merge 3 commits into
isaac-sim:mainfrom
chrisvoncsefalvay:feat/dvrk-needle-pass
Draft

Add dVRK needle-pass environment and OpenXR teleoperation#6527
chrisvoncsefalvay wants to merge 3 commits into
isaac-sim:mainfrom
chrisvoncsefalvay:feat/dvrk-needle-pass

Conversation

@chrisvoncsefalvay

@chrisvoncsefalvay chrisvoncsefalvay commented Jul 14, 2026

Copy link
Copy Markdown

Description

Part of the dVRK Extended Cinematic Universe: best read in conjunction with this PR.

This adds a contact-driven dVRK PSM needle-pass environment, its pinned PSM asset configuration, and OpenXR teleoperation support.

The environment starts with the needle physically retained in the donor jaws, drives transfer through measured jaw contact, and qualifies the hand-off without direct runtime needle-state writes or attachment shortcuts. It also hardens shared OpenXR controller validity handling and fixes the test dispatcher so Isaac Sim test failures propagate through CI.

The branch is intentionally a draft while NVIDIA/IsaacTeleop#769 remains open. CI installs its immutable merge revision (790d6cb4e948de377975c76ed1e9cbf5098e10fc) for the dVRK retargeter tests; that test-only pin should be replaced by the corresponding released dependency before this PR is merged.

Main changes

  • add a fixed-base dVRK PSM asset configuration and task-specific OpenXR device/retargeter
  • add Isaac-Needle-Pass-dVRK-v0 with differential IK, contact observations, load-backed hand-off phases, rewards, termination criteria, and CUDA-only validation
  • propagate OpenXR pose validity and command-line device overrides through teleoperation configurations
  • make CI load the Isaac Sim per-file dispatcher and return the real pytest status

Commit structure

  1. fix(openxr): harden controller tracking and validation
  2. feat(dvrk): add PSM asset and OpenXR teleoperation
  3. feat(tasks): add dVRK needle-pass environment

Validation

  • 72 focused OpenXR, dVRK device/retargeter, needle-pass, and CI-dispatcher tests passed
  • Isaac Sim 5.1.0 GPU physics qualification: 4 passed, 2 operator/hardware opt-in tests skipped
  • Ruff format and lint: 33 changed Python files passed
  • asset preflight: needle, suture pad, and dVRK PSM SHA-256 checks passed
  • git diff --check passed

Grasp-generation evidence addendum

The fixed donor and receiver grasps came from Isaac Sim 5.1.0 isaacsim.replicator.grasping 1.0.9 with seed 12: 256 centres by 32 orientations = 8,192 candidates. The selected, unmodified rows are donor 2321 and receiver 51. The retained evidence bundle is identified by SHA-256: asset 2b317a61f93631a7192e7ed2839ef20f7a75c05aa5f84a3905696134a64f36d7, wrapper 01bc820d1777a1655a5c42b3ebac997c6281335a12d12f7636c3e25721f3a2d5, configuration b308ec31bf9bf425c686007e0dc0ad72f09ae7e1f67e1015ac53dc92a017e798, candidates 7c601982d72759ca901fad9b59fa1df80a092221d1cb91eda88938b2b83bc374, and manifest 13c72a5fb58db7c211619b72dcbdf27890a25a35a5aa8e3185ab4ae3139970ee. The generation bundle is review evidence rather than installed package data. Happy to provide provenance.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Documentation update

Screenshots

download download

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new task warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in each affected extension's config/extension.toml file
  • I have added my name to CONTRIBUTORS.md

@github-actions github-actions Bot added documentation Improvements or additions to documentation asset New asset feature or request isaac-mimic Related to Isaac Mimic team infrastructure labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

asset New asset feature or request documentation Improvements or additions to documentation infrastructure isaac-mimic Related to Isaac Mimic team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant