Visuo-Tactile Feedback-Based Robot Manipulation for Object Packing

Page view(s)
Checked on Mar 26, 2024
Visuo-Tactile Feedback-Based Robot Manipulation for Object Packing
Visuo-Tactile Feedback-Based Robot Manipulation for Object Packing
Journal Title:
IEEE Robotics and Automation Letters
Publication Date:
13 January 2023
Liang, W., Fang, F., Acar, C., Toh, W. Q., Sun, Y., Xu, Q., & Wu, Y. (2023). Visuo-Tactile Feedback-Based Robot Manipulation for Object Packing. IEEE Robotics and Automation Letters, 8(2), 1151–1158.
Robots are increasingly expected to manipulate objects, of which properties have high perceptual uncertainty from any single sensory modality. This directly impacts successful object manipulation. Object packing is one of the challenging tasks in robot manipulation. In this work, a new visuo-tactile feedback-based manipulation planning framework for object packing is proposed, which makes use of the on-the-fly multisensory feedback and an attention-guided deep affordance model as perceptual states as well as a deep reinforcement learning (DRL) pipeline. Significantly, multiple sensory modalities, vision and touch [tactile and force/torque (F/T)], are employed in predicting and indicating the manipulable regions of multiple affordances (i.e., graspability and pushability) for objects with similar appearances but different intrinsic properties (e.g., mass distribution). To improve the manipulation efficiency, the DRL algorithm is trained to select the optimal actions for successful object manipulation. The proposed method is evaluated on both an open dataset and our collected dataset and demonstrated in the use case of the object packing task. The results show that the proposed method outperforms the existing methods and achieves better accuracy with much higher efficiency.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - AME Programmatic Funding Scheme
Grant Reference no. : A18A2b0046
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Files uploaded:

File Size Format Action
liang2023visuo-tactile.pdf 5.64 MB PDF Request a copy