We are investigating natural haptic teleoperation for on-site assembly of long-span buildings. (a) The envisioned interaction scenario. (b) An operator wearing an inertial motion-capture suit. (c) A Baxter robot using our retargeting algorithm to follow the operator's arm motions. (d) Our audio-based haptic feedback system and the setup for a user study on construction using a da Vinci robot. (e) A three-axis accelerometer mounted on a robotic tool. (f) A haptic actuator attached to the handle. (g) Construction toys assembled in the study.
Construction machines are usually controlled through joint-space interfaces that are not intuitive or dexterous. In the context of the IntCDC Excellence Cluster, we envision allowing teleoperation of such machines through natural arm motions, and we seek an effective and robust method for providing haptic feedback to facilitate teleoperated construction activities. As such, we are developing an advanced telerobotic system with haptic perception for operators in on-site construction. It has two subsystems: a motion-mapping interface and a wearable vibrotactile feedback system.
We believe the operator should be able to control a construction robot through natural arm motions. We are thus developing a motion-mapping interface that senses the operator's arm movements and commands the robot's arm to move in a similar way. The operator's arm is tracked using an Xsens Link, an inertial motion-capture system. The algorithm is formulated as a constrained optimization problem that minimizes the retargeting error, defined as the error between the pose of the human and that of the robot. We used a Baxter Research Robot as our platform for testing the initial prototype of this algorithm. We are working to adapt the algorithm for use with the IntCDC mini-crane in combination with our haptic feedback system.
The wearable vibrotactile feedback system enables the operator to feel the physical vibrations of the teleoperated machine as it moves and makes contact with other objects on the construction site, improving on our haptic feedback approach from robotic surgery. Our robust system uses accelerometers, an audio mixer, a stereo audio amplifier, and haptic actuators. As an initial validation, we ran a user study using a wired version on a da Vinci robot. Participants performed an assembly task with construction toys in three haptic feedback conditions: no vibrations, one-axis vibrations, and three-axis vibrations. The quantitative and qualitative measures were significantly improved when participants received haptic feedback.