AgentsAutonomy & tool use
NVIDIA Shows AI Agents Building Simulation Assets to Train Pollen Robotics' Microduck to Walk
NVIDIA has published a tutorial in which AI agents running on an RTX Spark PC plan a robot-training project and build 3D obstacle-course assets in FreeCAD and Blender, which then feed Isaac Lab locomotion training for Pollen Robotics' $400 open-source Microduck.
From the NVIDIA Generative AI article of October 23, 2026 and cited sources: The Hermes agent plans the project using Nemotron 3.5 Bolt, an open-weight mixture-of-experts model with 74B total and 9B active parameters, sized to fit in 24GB. [1] [5]
SOURCES[1] Using AI Agents to Teach Robots Like Microduck to Walk in… developer.nvidia.com[5] Nemotron 3.5 Bolt model card huggingface.co
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- The Hermes agent plans the project using Nemotron 3.5 Bolt, an open-weight mixture-of-experts model with 74B total and 9B active parameters, sized to fit in 24GB. [1] [5]
- Original source
- NVIDIA Generative AI article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 23, 2026 article published
- Covered
- Oct 25 edition
- Evidence
- 8 sources across 7 sites
- 1Using AI Agents to Teach Robots Like Microduck to Walk in…developer.nvidia.com
- 2Microduck Isaac Lab sim-to-real tutorial (Hugging Face)huggingface.co
- 3NVIDIA and Microsoft RTX Spark press releaseinvestor.nvidia.com
- 4Microducks Exude Macro Charm (Make:)makezine.com
- 5Nemotron 3.5 Bolt model cardhuggingface.co
- 6Better Actuator Models documentationbam.readthedocs.io
Reach: not measured yet
The Hermes agent plans the project using Nemotron 3.5 Bolt, an open-weight mixture-of-experts model with 74B total and 9B active parameters, sized to fit in 24GB. A second agent, Astra, drives FreeCAD and Blender to turn a generated concept image into USD assets. When a dimension cannot be seen in the image, the agent records it as a modeling assumption. [1] [5]
The agent-built course feeds an existing training stack. A pretrained locomotion policy was fine-tuned for ramps and uneven terrain using depth images and robot-state inputs, across 4,096 parallel environments on one RTX PRO 6000 GPU. The base setup uses Isaac Lab 3.0, the Newton physics engine, BAM actuator models and domain randomization. NVIDIA reports training speeds of about 1.25M steps per second. [1] [2] [6]
The obstacle-course policy is checked only qualitatively. The post gives no completion, fall or collision rates and does not publish its reward terms, prompts or generated assets. The policy was not run on a physical robot; the real-robot clip shows a separate locomotion policy. [1]
The tutorial shows agents building simulation environments and assets for robot training. If that holds up, small teams could prototype locomotion training faster.
Read the full assessment
Reward design and sim-to-real calibration are still manual, and the quality of the agents' output is unmeasured.
Executive brief
NVIDIA says a prompt typed on an RTX Spark PC can kick off most of a robot-training pipeline. AI agents write the project plan and build 3D obstacle-course assets in FreeCAD and Blender, and those assets go into the training environment for Microduck, Pollen Robotics' $400 open-source duck robot. The NVIDIA blog post (Oct 23, 2026) is a tutorial. It reports no course-completion rates, fall counts or other policy metrics. No independent party has reproduced the workflow, and none of the reviewed sources corroborate it. The training numbers it does give (4,096 parallel environments, about 1.25M steps/s) come from NVIDIA's own Hugging Face tutorial.
What changed and event timeline
RTX Spark announced
NVIDIA and Microsoft introduced Windows PCs built on an Arm Grace CPU plus a Blackwell GPU, with up to 128GB unified memory, shipping Fall 2026.
Microduck pre-orders open
Pollen Robotics and Hugging Face opened orders for the roughly $400 trainable duck robot, which descends from Disney's BDX droid and the Open Duck Mini project. First deliveries are targeted before Christmas.
Nemotron 3.5 Bolt published
NVIDIA released a 74B-total, 9B-active MoE model sized to fit in 24GB of memory. It is the planning model used in the workflow.
Sim-to-real tutorial
NVIDIA staff documented Microduck locomotion training in Isaac Lab 3.0 with Newton, BAM actuator models and domain randomization.
Agentic workflow post
NVIDIA added agent-driven planning and asset creation on top of that training stack, plus a perception step that uses the head-camera depth image.
Capabilities and access
- Planning: the Hermes agent running Nemotron 3.5 Bolt 74B-A9B-24GB, which has open weights under OpenMDW-1.1 and a 256K-token context.
- Asset creation: "Astra" drives FreeCAD and Blender and exports USD files. None of the reviewed sources document Astra outside the post.
- Simulation and training: Isaac Sim, the open-source Isaac Lab Microduck task, the Newton physics engine, and SimReady assets (more than 2,000, free).
Read the full section
- Planning: the Hermes agent running Nemotron 3.5 Bolt 74B-A9B-24GB, which has open weights under OpenMDW-1.1 and a 256K-token context.
- Asset creation: "Astra" drives FreeCAD and Blender and exports USD files. None of the reviewed sources document Astra outside the post.
- Simulation and training: Isaac Sim, the open-source Isaac Lab Microduck task, the Newton physics engine, and SimReady assets (more than 2,000, free).
- Platform caveat: Isaac Sim runs on RTX Spark as Windows x86 binaries under Arm emulation (post).
Technical analysis for researchers and developers
- Agent pipeline: the agent analyses a generated concept image and builds an asset inventory, recording parts that are cut off or hidden.
- Policy: a pretrained locomotion policy was fine-tuned for ramps and uneven terrain using depth and robot-state observations, across 4,096 environments on one RTX PRO 6000 Blackwell GPU.
- Base stack: 61 observations in, 14 joint-position targets out. Training uses RSL-RL with Newton's MJWarp solver.
Read the full section
- Agent pipeline: the agent analyses a generated concept image and builds an asset inventory, recording parts that are cut off or hidden. A cloud tool enhances each crop, and dimensions that can't be seen are logged as modeling assumptions (post).
- Policy: a pretrained locomotion policy was fine-tuned for ramps and uneven terrain using depth and robot-state observations, across 4,096 environments on one RTX PRO 6000 Blackwell GPU.
- Base stack: 61 observations in, 14 joint-position targets out. Training uses RSL-RL with Newton's MJWarp solver. Domain randomization covers mass, friction, encoder offsets, voltage sag, latency and per-joint backlash (HF tutorial).
- Reproducibility gaps: the post does not publish the reward terms, training time, prompts or the generated assets.
Claims and evidence
- Agents turn a prompt into a plan and usable USD assets. This is shown only through figures in NVIDIA's post.
- The trained policy completes the obstacle course. Validation is described only in qualitative terms, and no success rates are reported (post).
- Training reaches up to 1.25M steps/s and a physical robot recovers from being knocked over.
Read the full section
- Agents turn a prompt into a plan and usable USD assets. This is shown only through figures in NVIDIA's post. The post gives no measure of asset accuracy or how often the agents failed.
- The trained policy completes the obstacle course. Validation is described only in qualitative terms, and no success rates are reported (post).
- Training reaches up to 1.25M steps/s and a physical robot recovers from being knocked over. Both claims are NVIDIA-reported and backed by demo media rather than benchmarks (HF tutorial).
- BAM improves actuator fidelity. This rests on peer-reviewed academic work, an ICRA 2025 paper by Duclusaud et al. (BAM docs).
Context and prior work
- Microduck comes from Disney's BDX droid via the community Open Duck Mini project. Its SDK and reinforcement-learning stack are open on GitHub (Make:, eesel overview).
- BAM was developed by researchers at Rhoban (Bordeaux). Before the Isaac Lab/Newton port, it targeted MuJoCo and mjlab (BAM docs, Rhoban TDP).
Read the full section
- Microduck comes from Disney's BDX droid via the community Open Duck Mini project. Its SDK and reinforcement-learning stack are open on GitHub (Make:, eesel overview).
- BAM was developed by researchers at Rhoban (Bordeaux). It adds extended servo friction models (M1–M6) that capture effects the standard Coulomb-viscous model misses, such as Stribeck friction and load dependence. Before the Isaac Lab/Newton port, it targeted MuJoCo and mjlab (BAM docs, Rhoban TDP).
Limitations, safety and contested findings
- Scope: physical deployment of the obstacle-course policy is explicitly out of scope. The real-robot clip shows a separate locomotion policy (post).
- Internal inconsistency: the post describes Isaac Lab training as running "in the cloud," but the reported run used a single workstation GPU.
- Safety guardrail: the agent prompt tells the agents not to operate physical hardware.
Read the full section
- Scope: physical deployment of the obstacle-course policy is explicitly out of scope. The real-robot clip shows a separate locomotion policy (post).
- Internal inconsistency: the post describes Isaac Lab training as running "in the cloud," but the reported run used a single workstation GPU.
- Safety guardrail: the agent prompt tells the agents not to operate physical hardware.
- Agent reliability: the post's own checks allow assets to end up "demonstrably blocked." That means the agents can fail at asset creation, and the post gives no failure rate.
Business and practitioner implications
- Cost of entry: a $400 robot, open training code and free assets lower the barrier for teaching and prototyping.
- Where agents help: they mainly speed up building the environment and its assets. Reward design and sim-to-real calibration remain manual engineering work.
- Evaluation discipline: teams should log completion, fall and collision rates themselves, as the post recommends, because NVIDIA publishes none.
Read the full section
- Cost of entry: a $400 robot, open training code and free assets lower the barrier for teaching and prototyping. The workflow still assumes access to an RTX Spark and an RTX PRO 6000-class GPU.
- Where agents help: they mainly speed up building the environment and its assets. Reward design and sim-to-real calibration remain manual engineering work.
- Evaluation discipline: teams should log completion, fall and collision rates themselves, as the post recommends, because NVIDIA publishes none.
- Platform lock-in: the pipeline ties together NVIDIA hardware, models and simulators. MuJoCo/BAM is a documented alternative path (BAM docs).