Haptic Intelligence

Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

2023

Article

al

hi


Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor's sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290,000 simulated and 90,000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline's results outperform predictions by both a physics-based model and end-to-end learning.

Author(s): Hyosang Lee and Huanbo Sun and Hyunkyu Park and Gokhan Serhat and Bernard Javot and Georg Martius and Katherine J. Kuchenbecker
Journal: IEEE Transactions on Automation Science and Engineering
Volume: 20
Number (issue): 1
Pages: 425--439
Year: 2023
Month: January

Department(s): Autonomous Learning, Haptic Intelligence
Research Project(s): Efficient Large-Area Tactile Sensing for Robot Skin
Bibtex Type: Article (article)
Paper Type: Journal

DOI: 10.1109/TASE.2022.3156184
State: Published

BibTex

@article{Lee23-TASE-Map,
  title = {Predicting the Force Map of an {ERT}-Based Tactile Sensor Using Simulation and Deep Networks},
  author = {Lee, Hyosang and Sun, Huanbo and Park, Hyunkyu and Serhat, Gokhan and Javot, Bernard and Martius, Georg and Kuchenbecker, Katherine J.},
  journal = {IEEE Transactions on Automation Science and Engineering},
  volume = {20},
  number = {1},
  pages = {425--439},
  month = jan,
  year = {2023},
  doi = {10.1109/TASE.2022.3156184},
  month_numeric = {1}
}