Sim2Real Transfer of Manipulator Tasks using Diffusion Models and ACTs
Date:
Cannot Release Code and Images due to company policies.
Sim2Real Framework Development: Built a robust Sim2Real transfer pipeline to enable effective deployment of pick-and-place tasks from simulation to a physical UR10 manipulator. Leveraged Action Chunking Transformers (ACTs): Designed and implemented ACTs to efficiently model and predict action sequences, capturing temporal dependencies in robotic manipulation tasks. Custom Gripper Design: Developed a 3D-printed gripper model from scratch in MuJoCo, ensuring accurate contact dynamics and interaction modeling in the simulation environment. Feature Extraction Optimization: Engineered a custom feature extractor integrated with the transformer architecture to preprocess spatial and force-related features critical for action chunking. Simulation Using MuJoCo: Utilized the MuJoCo Menagerie package to simulate the UR10 manipulator with high fidelity, ensuring realistic physics and control dynamics. Improved Skill Transfer: Validated ACT-based models for their effectiveness in transferring learned policies to the real robot, ensuring precision and consistency in manipulation tasks. Advanced Algorithm Implementation: Incorporated Diffusion Policies alongside ACTs to refine the policy generation for robust and adaptive skill learning. Collaboration and Iterative Testing: Conducted iterative testing between simulation and real-world scenarios to fine-tune the ACT-based Sim2Real framework for optimal performance.