Zehang Weng

Cite

Citations (0)

DexDiffuser: Generating Dexterous Grasps with Diffusion Models

arXiv (Cornell University) (2024)

Zehang Weng Haofei Lu Danica Kragić Jens Lundell

We introduce DexDiffuser, a novel dexterous grasping method that generates, evaluates, and refines grasps on partial object point clouds. DexDiffuser includes the conditional diffusion-based grasp sampler DexSampler and the dexterous grasp evaluator DexEvaluator. DexSampler generates high-quality grasps conditioned on object point clouds by iterative denoising of randomly sampled grasps. We also introduce two grasp refinement strategies: Evaluator-Guided Diffusion (EGD) and Evaluator-based Sampling Refinement (ESR). Our simulation and real-world experiments on the Allegro Hand consistently demonstrate that DexDiffuser outperforms the state-of-the-art multi-finger grasp generation method FFHNet with an, on average, 21.71--22.20\% higher grasp success rate.

10.48550/arxiv.2402.02989

Cite

Citations (0)

CAPGrasp: An $\mathbb {R}^{3}\times \text{SO(2)-Equivariant}$ Continuous Approach-Constrained Generative Grasp Sampler

IEEE Robotics and Automation Letters (2024)

Zehang Weng Haofei Lu Jens Lundell Danica Kragić

Equivariant map

10.1109/lra.2024.3369444

Cite

Citations (0)

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2021)

Zehang Weng Fabian Paus Anastasiia Varava Hang Yin Tamim Asfour

Capturing scene dynamics and predicting the future scene state is challenging but essential for robotic manipulation tasks, especially when the scene contains both rigid and deformable objects. In this work, we contribute a simulation environment and generate a novel dataset for task-specific manipulation, involving interactions between rigid objects and a deformable bag. The dataset incorporates a rich variety of scenarios including different object sizes, object numbers and manipulation actions. We approach dynamics learning by proposing an object-centric graph representation and two modules which are Active Prediction Module (APM) and Position Prediction Module (PPM) based on graph neural networks with an encode-process-decode architecture. At the inference stage, we build a two-stage model based on the learned modules for single time step prediction. We combine modules with different prediction horizons into a mixed-horizon model which addresses long-term prediction. In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks. Supplementary material is available at https://github.com/wengzehang/deformable_rigid_interaction_prediction

ENCODE

Representation

10.1109/iros51168.2021.9636660

Cite

Citations (11)

Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images

Lecture notes in computer science (2024)

Ci Li Yi Yang Zehang Weng Elin Hernlund Silvia Zuffi

10.1007/978-981-96-0972-7_16

Cite

Citations (0)

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

arXiv (Cornell University) (2021)

Zehang Weng Fabian Paus Anastasiia Varava Hang Yin Tamim Asfour

Capturing scene dynamics and predicting the future scene state is challenging but essential for robotic manipulation tasks, especially when the scene contains both rigid and deformable objects. In this work, we contribute a simulation environment and generate a novel dataset for task-specific manipulation, involving interactions between rigid objects and a deformable bag. The dataset incorporates a rich variety of scenarios including different object sizes, object numbers and manipulation actions. We approach dynamics learning by proposing an object-centric graph representation and two modules which are Active Prediction Module (APM) and Position Prediction Module (PPM) based on graph neural networks with an encode-process-decode architecture. At the inference stage, we build a two-stage model based on the learned modules for single time step prediction. We combine modules with different prediction horizons into a mixed-horizon model which addresses long-term prediction. In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks. Supplementary material is available at https://github.com/wengzehang/deformable_rigid_interaction_prediction

Representation

ENCODE

10.48550/arxiv.2103.02932

Cite

Citations (2)

Dynamic Environments with Deformable Objects

Neural Information Processing Systems (2021)

Rika Antonova Peiyang Shi Hang Yin Zehang Weng Danica Kragic Jensfelt

Source

Cite

Citations (2)

GoNet: An Approach-Constrained Generative Grasp Sampling Network

Zehang Weng Haofei Lu Jens Lundell Danica Kragić

This work addresses the problem of learning approach-constrained data-driven grasp samplers. To this end, we propose GoNet: a generative grasp sampler that can constrain the grasp approach direction to a subset of SO(3). The key insight is to discretize SO(3) into a predefined number of bins and train GoNet to generate grasps whose approach directions are within those bins. At run-time, the bin aligning with the second largest principal component of the observed point cloud is selected. GoNet is benchmarked against GraspNet, a state-of-the-art unconstrained grasp sampler, in an unconfined grasping experiment in simulation and on an unconfined and confined grasping experiment in the real world. The results demonstrate that GoNet achieves higher success-over-coverage in simulation and a 12%-18% higher success rate in real-world table-picking and shelf-picking tasks than the baseline.

Table (database)

Generative model

Baseline (sea)

10.1109/humanoids57100.2023.10375235

Cite

Citations (2)

Interactive Perception for Deformable Object Manipulation

arXiv (Cornell University) (2024)

Zehang Weng Peng Zhou Hang Yin Alexander Kravberg Anastasiia Varava

Interactive perception enables robots to manipulate the environment and objects to bring them into states that benefit the perception process. Deformable objects pose challenges to this due to significant manipulation difficulty and occlusion in vision-based perception. In this work, we address such a problem with a setup involving both an active camera and an object manipulator. Our approach is based on a sequential decision-making framework and explicitly considers the motion regularity and structure in coupling the camera and manipulator. We contribute a method for constructing and computing a subspace, called Dynamic Active Vision Space (DAVS), for effectively utilizing the regularity in motion exploration. The effectiveness of the framework and approach are validated in both a simulation and a real dual-arm robot setup. Our results confirm the necessity of an active camera and coordinative motion in interactive perception for deformable objects.

10.48550/arxiv.2403.05177

Cite

Citations (0)

GoNet: An Approach-Constrained Generative Grasp Sampling Network

arXiv (Cornell University) (2023)

Zehang Weng Haofei Lu Jens Lundell Danica Kragić

This work addresses the problem of learning approach-constrained data-driven grasp samplers. To this end, we propose GoNet: a generative grasp sampler that can constrain the grasp approach direction to a subset of SO(3). The key insight is to discretize SO(3) into a predefined number of bins and train GoNet to generate grasps whose approach directions are within those bins. At run-time, the bin aligning with the second largest principal component of the observed point cloud is selected. GoNet is benchmarked against GraspNet, a state-of-the-art unconstrained grasp sampler, in an unconfined grasping experiment in simulation and on an unconfined and confined grasping experiment in the real world. The results demonstrate that GoNet achieves higher success-over-coverage in simulation and a 12%-18% higher success rate in real-world table-picking and shelf-picking tasks than the baseline.

Baseline (sea)

Table (database)

Generative model

10.48550/arxiv.2303.07972

Cite

Citations (0)

Papers

Citation

H-Index

i-10 index

Papers

Citation

H-Index

i-10 index