Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks
2021
This paper introduces a discrete-continuous action space to learn insertion
primitives for robotic assembly tasks. Primitive is a sequence of elementary
actions with certain exit conditions, such as "pushing down the peg until
contact". Since the primitive is an abstraction of robot control commands and
encodes human prior knowledge, it reduces the exploration difficulty and yields
better learning efficiency. In this paper, we learn robot assembly skills via
primitives. Specifically, we formulate insertion primitives as parameterized
actions: hybrid actions consisting of discrete primitive types and continuous
primitive parameters. Compared with the previous work using a set of
discretized parameters for each primitive, the agent in our method can freely
choose primitive parameters from a continuous space, which is more flexible and
efficient. To learn these insertion primitives, we propose Twin-Smoothed
Multi-pass Deep Q-Network (TS-MP-DQN), an advanced version of MP-DQN with twin
Q-network to reduce the Q-value over-estimation. Extensive experiments are
conducted in the simulation and real world for validation. From experiment
results, our approach achieves higher success rates than three baselines:
MP-DQN with parameterized actions, primitives with discrete parameters, and
continuous velocity control. Furthermore, learned primitives are robust to
sim-to-real transfer and can generalize to challenging assembly tasks such as
tight round peg-hole and complex shaped electric connectors with promising
success rates. Experiment videos are available at
https://msc.berkeley.edu/research/insertion-primitives.html.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
13
References
0
Citations
NaN
KQI