Haptic Intelligence


2024


Being Neurodivergent in Academia: Autistic and abroad
Being Neurodivergent in Academia: Autistic and abroad

Schulz, A.

eLife, 13, March 2024 (article)

Abstract
An AuDHD researcher recounts the highs and lows of relocating from the United States to Germany for his postdoc.

DOI [BibTex]


{IMU}-Based Kinematics Estimation Accuracy Affects Gait Retraining Using Vibrotactile Cues
IMU-Based Kinematics Estimation Accuracy Affects Gait Retraining Using Vibrotactile Cues

Rokhmanova, N., Pearl, O., Kuchenbecker, K. J., Halilaj, E.

IEEE Transactions on Neural Systems and Rehabilitation Engineering, 32, pages: 1005-1012, February 2024 (article)

Abstract
Wearable sensing using inertial measurement units (IMUs) is enabling portable and customized gait retraining for knee osteoarthritis. However, the vibrotactile feedback that users receive directly depends on the accuracy of IMU-based kinematics. This study investigated how kinematic errors impact an individual's ability to learn a therapeutic gait using vibrotactile cues. Sensor accuracy was computed by comparing the IMU-based foot progression angle to marker-based motion capture, which was used as ground truth. Thirty subjects were randomized into three groups to learn a toe-in gait: one group received vibrotactile feedback during gait retraining in the laboratory, another received feedback outdoors, and the control group received only verbal instruction and proceeded directly to the evaluation condition. All subjects were evaluated on their ability to maintain the learned gait in a new outdoor environment. We found that subjects with high tracking errors exhibited more incorrect responses to vibrotactile cues and slower learning rates than subjects with low tracking errors. Subjects with low tracking errors outperformed the control group in the evaluation condition, whereas those with higher error did not. Errors were correlated with foot size and angle magnitude, which may indicate a non-random algorithmic bias. The accuracy of IMU-based kinematics has a cascading effect on feedback; ignoring this effect could lead researchers or clinicians to erroneously classify a patient as a non-responder if they did not improve after retraining. To use patient and clinician time effectively, future implementation of portable gait retraining will require assessment across a diverse range of patients.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
How Should Robots Exercise with People? Robot-Mediated Exergames Win with Music, Social Analogues, and Gameplay Clarity

Fitter, N. T., Mohan, M., Preston, R. C., Johnson, M. J., Kuchenbecker, K. J.

Frontiers in Robotics and AI, 10(1155837):1-18, January 2024 (article)

Abstract
The modern worldwide trend toward sedentary behavior comes with significant health risks. An accompanying wave of health technologies has tried to encourage physical activity, but these approaches often yield limited use and retention. Due to their unique ability to serve as both a health-promoting technology and a social peer, we propose robots as a game-changing solution for encouraging physical activity. This article analyzes the eight exergames we previously created for the Rethink Baxter Research Robot in terms of four key components that are grounded in the video-game literature: repetition, pattern matching, music, and social design. We use these four game facets to assess gameplay data from 40 adult users who each experienced the games in balanced random order. In agreement with prior research, our results show that relevant musical cultural references, recognizable social analogues, and gameplay clarity are good strategies for taking an otherwise highly repetitive physical activity and making it engaging and popular among users. Others who study socially assistive robots and rehabilitation robotics can benefit from this work by considering the presented design attributes to generate future hypotheses and by using our eight open-source games to pursue follow-up work on social-physical exercise with robots.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
Robust Surface Recognition with the Maximum Mean Discrepancy: Degrading Haptic-Auditory Signals through Bandwidth and Noise

Khojasteh, B., Shao, Y., Kuchenbecker, K. J.

IEEE Transactions on Haptics, pages: 1-8, January 2024 (article)

Abstract
Sliding a tool across a surface generates rich sensations that can be analyzed to recognize what is being touched. However, the optimal configuration for capturing these signals is yet unclear. To bridge this gap, we consider haptic-auditory data as a human explores surfaces with different steel tools, including accelerations of the tool and finger, force and torque applied to the surface, and contact sounds. Our classification pipeline uses the maximum mean discrepancy (MMD) to quantify differences in data distributions in a high-dimensional space for inference. With recordings from three hemispherical tool diameters and ten diverse surfaces, we conducted two degradation studies by decreasing sensing bandwidth and increasing added noise. We evaluate the haptic-auditory recognition performance achieved with the MMD to compare newly gathered data to each surface in our known library. The results indicate that acceleration signals alone have great potential for high-accuracy surface recognition and are robust against noise contamination. The optimal accelerometer bandwidth exceeds 1000 Hz, suggesting that useful vibrotactile information extends beyond human perception range. Finally, smaller tool tips generate contact vibrations with better noise robustness. The provided sensing guidelines may enable superhuman performance in portable surface recognition, which could benefit quality control, material documentation, and robotics.

DOI Project Page [BibTex]

2023


no image
Towards Semi-Automated Pleural Cavity Access for Pneumothorax in Austere Environments

L’Orsa, R., Lama, S., Westwick, D., Sutherland, G., Kuchenbecker, K. J.

Acta Astronautica, 212, pages: 48-53, November 2023 (article)

Abstract
Astronauts are at risk for pneumothorax, a condition where injury or disease introduces air between the chest wall and the lungs (i.e., the pleural cavity). In a worst-case scenario, it can rapidly lead to a fatality if left unmanaged and will require prompt treatment in situ if developed during spaceflight. Chest tube insertion is the definitive treatment for pneumothorax, but it requires a high level of skill and frequent practice for safe use. Physician astronauts may struggle to maintain this skill on medium- and long-duration exploration-class missions, and it is inappropriate for pure just-in-time learning or skill refreshment paradigms. This paper proposes semi-automating tool insertion to reduce the risk of complications in austere environments and describes preliminary experiments providing initial validation of an intelligent prototype system. Specifically, we showcase and analyse motion and force recordings from a sensorized percutaneous access needle inserted repeatedly into an ex vivo tissue phantom, along with relevant physiological data simultaneously recorded from the operator. When coupled with minimal just-in-time training and/or augmented reality guidance, the proposed system may enable non-expert operators to safely perform emergency chest tube insertion without the use of ground resources.

DOI Project Page [BibTex]

2023

DOI Project Page [BibTex]


no image
Multimodal Multi-User Surface Recognition with the Kernel Two-Sample Test

Khojasteh, B., Solowjow, F., Trimpe, S., Kuchenbecker, K. J.

IEEE Transactions on Automation Science and Engineering, pages: 1-16, August 2023 (article)

Abstract
Machine learning and deep learning have been used extensively to classify physical surfaces through images and time-series contact data. However, these methods rely on human expertise and entail the time-consuming processes of data and parameter tuning. To overcome these challenges, we propose an easily implemented framework that can directly handle heterogeneous data sources for classification tasks. Our data-versus-data approach automatically quantifies distinctive differences in distributions in a high-dimensional space via kernel two-sample testing between two sets extracted from multimodal data (e.g., images, sounds, haptic signals). We demonstrate the effectiveness of our technique by benchmarking against expertly engineered classifiers for visual-audio-haptic surface recognition due to the industrial relevance, difficulty, and competitive baselines of this application; ablation studies confirm the utility of key components of our pipeline. As shown in our open-source code, we achieve 97.2% accuracy on a standard multi-user dataset with 108 surface classes, outperforming the state-of-the-art machine-learning algorithm by 6% on a more difficult version of the task. The fact that our classifier obtains this performance with minimal data processing in the standard algorithm setting reinforces the powerful nature of kernel methods for learning to recognize complex patterns. Note to Practitioners—We demonstrate how to apply the kernel two-sample test to a surface-recognition task, discuss opportunities for improvement, and explain how to use this framework for other classification problems with similar properties. Automating surface recognition could benefit both surface inspection and robot manipulation. Our algorithm quantifies class similarity and therefore outputs an ordered list of similar surfaces. This technique is well suited for quality assurance and documentation of newly received materials or newly manufactured parts. More generally, our automated classification pipeline can handle heterogeneous data sources including images and high-frequency time-series measurements of vibrations, forces and other physical signals. As our approach circumvents the time-consuming process of feature engineering, both experts and non-experts can use it to achieve high-accuracy classification. It is particularly appealing for new problems without existing models and heuristics. In addition to strong theoretical properties, the algorithm is straightforward to use in practice since it requires only kernel evaluations. Its transparent architecture can provide fast insights into the given use case under different sensing combinations without costly optimization. Practitioners can also use our procedure to obtain the minimum data-acquisition time for independent time-series data from new sensor recordings.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Minsight: A Fingertip-Sized Vision-Based Tactile Sensor for Robotic Manipulation
Minsight: A Fingertip-Sized Vision-Based Tactile Sensor for Robotic Manipulation

Andrussow, I., Sun, H., Kuchenbecker, K. J., Martius, G.

Advanced Intelligent Systems, 5(8):2300042, August 2023, Inside back cover (article)

Abstract
Intelligent interaction with the physical world requires perceptual abilities beyond vision and hearing; vibrant tactile sensing is essential for autonomous robots to dexterously manipulate unfamiliar objects or safely contact humans. Therefore, robotic manipulators need high-resolution touch sensors that are compact, robust, inexpensive, and efficient. The soft vision-based haptic sensor presented herein is a miniaturized and optimized version of the previously published sensor Insight. Minsight has the size and shape of a human fingertip and uses machine learning methods to output high-resolution maps of 3D contact force vectors at 60 Hz. Experiments confirm its excellent sensing performance, with a mean absolute force error of 0.07 N and contact location error of 0.6 mm across its surface area. Minsight's utility is shown in two robotic tasks on a 3-DoF manipulator. First, closed-loop force control enables the robot to track the movements of a human finger based only on tactile data. Second, the informative value of the sensor output is shown by detecting whether a hard lump is embedded within a soft elastomer with an accuracy of 98%. These findings indicate that Minsight can give robots the detailed fingertip touch sensing needed for dexterous manipulation and physical human–robot interaction.

DOI Project Page [BibTex]


Learning to Estimate Palpation Forces in Robotic Surgery From Visual-Inertial Data
Learning to Estimate Palpation Forces in Robotic Surgery From Visual-Inertial Data

Lee, Y., Husin, H. M., Forte, M., Lee, S., Kuchenbecker, K. J.

IEEE Transactions on Medical Robotics and Bionics, 5(3):496-506, August 2023 (article)

Abstract
Surgeons cannot directly touch the patient's tissue in robot-assisted minimally invasive procedures. Instead, they must palpate using instruments inserted into the body through trocars. This way of operating largely prevents surgeons from using haptic cues to localize visually undetectable structures such as tumors and blood vessels, motivating research on direct and indirect force sensing. We propose an indirect force-sensing method that combines monocular images of the operating field with measurements from IMUs attached externally to the instrument shafts. Our method is thus suitable for various robotic surgery systems as well as laparoscopic surgery. We collected a new dataset using a da Vinci Si robot, a force sensor, and four different phantom tissue samples. The dataset includes 230 one-minute-long recordings of repeated bimanual palpation tasks performed by four lay operators. We evaluated several network architectures and investigated the role of the network inputs. Using the DenseNet vision model and including inertial data best-predicted palpation forces (lowest average root-mean-square error and highest average coefficient of determination). Ablation studies revealed that video frames carry significantly more information than inertial signals. Finally, we demonstrated the model's ability to generalize to unseen tissue and predict shear contact forces.

DOI [BibTex]

DOI [BibTex]


Generating Clear Vibrotactile Cues with a Magnet Embedded in a Soft Finger Sheath
Generating Clear Vibrotactile Cues with a Magnet Embedded in a Soft Finger Sheath

Gertler, I., Serhat, G., Kuchenbecker, K. J.

Soft Robotics, 10(3):624-635, June 2023 (article)

Abstract
Haptic displays act on the user's body to stimulate the sense of touch and enrich applications from gaming and computer-aided design to rehabilitation and remote surgery. However, when crafted from typical rigid robotic components, they tend to be heavy, bulky, and expensive, while sleeker designs often struggle to create clear haptic cues. This article introduces a lightweight wearable silicone finger sheath that can deliver salient and rich vibrotactile cues using electromagnetic actuation. We fabricate the sheath on a ferromagnetic mandrel with a process based on dip molding, a robust fabrication method that is rarely used in soft robotics but is suitable for commercial production. A miniature rare-earth magnet embedded within the silicone layers at the center of the finger pad is driven to vibrate by the application of alternating current to a nearby air-coil. Experiments are conducted to determine the amplitude of the magnetic force and the frequency response function for the displacement amplitude of the magnet perpendicular to the skin. In addition, high-fidelity finite element analyses of the finger wearing the device are performed to investigate the trends observed in the measurements. The experimental and simulated results show consistent dynamic behavior from 10 to 1000 Hz, with the displacement decreasing after about 300 Hz. These results match the detection threshold profile obtained in a psychophysical study performed by 17 users, where more current was needed only at the highest frequency. A cue identification experiment and a demonstration in virtual reality validate the feasibility of this approach to fingertip haptics.

DOI Project Page [BibTex]


In the Arms of a Robot: Designing Autonomous Hugging Robots with Intra-Hug Gestures
In the Arms of a Robot: Designing Autonomous Hugging Robots with Intra-Hug Gestures

Block, A. E., Seifi, H., Hilliges, O., Gassert, R., Kuchenbecker, K. J.

ACM Transactions on Human-Robot Interaction, 12(2):1-49, June 2023, Special Issue on Designing the Robot Body: Critical Perspectives on Affective Embodied Interaction (article)

Abstract
Hugs are complex affective interactions that often include gestures like squeezes. We present six new guidelines for designing interactive hugging robots, which we validate through two studies with our custom robot. To achieve autonomy, we investigated robot responses to four human intra-hug gestures: holding, rubbing, patting, and squeezing. Thirty-two users each exchanged and rated sixteen hugs with an experimenter-controlled HuggieBot 2.0. The robot's inflated torso's microphone and pressure sensor collected data of the subjects' demonstrations that were used to develop a perceptual algorithm that classifies user actions with 88% accuracy. Users enjoyed robot squeezes, regardless of their performed action, they valued variety in the robot response, and they appreciated robot-initiated intra-hug gestures. From average user ratings, we created a probabilistic behavior algorithm that chooses robot responses in real time. We implemented improvements to the robot platform to create HuggieBot 3.0 and then validated its gesture perception system and behavior algorithm with sixteen users. The robot's responses and proactive gestures were greatly enjoyed. Users found the robot more natural, enjoyable, and intelligent in the last phase of the experiment than in the first. After the study, they felt more understood by the robot and thought robots were nicer to hug.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Haptify: A Measurement-Based Benchmarking System for Grounded Force-Feedback Devices
Haptify: A Measurement-Based Benchmarking System for Grounded Force-Feedback Devices

Fazlollahi, F., Kuchenbecker, K. J.

IEEE Transactions on Robotics, 39(2):1622-1636, April 2023 (article)

Abstract
Grounded force-feedback (GFF) devices are an established and diverse class of haptic technology based on robotic arms. However, the number of designs and how they are specified make comparing devices difficult. We thus present Haptify, a benchmarking system that can thoroughly, fairly, and noninvasively evaluate GFF haptic devices. The user holds the instrumented device end-effector and moves it through a series of passive and active experiments. Haptify records the interaction between the hand, device, and ground with a seven-camera optical motion-capture system, a 60-cm-square custom force plate, and a customized sensing end-effector. We demonstrate six key ways to assess GFF device performance: workspace shape, global free-space forces, global free-space vibrations, local dynamic forces and torques, frictionless surface rendering, and stiffness rendering. We then use Haptify to benchmark two commercial haptic devices. With a smaller workspace than the 3D Systems Touch, the more expensive Touch X outputs smaller free-space forces and vibrations, smaller and more predictable dynamic forces and torques, and higher-quality renderings of a frictionless surface and high stiffness.

DOI Project Page [BibTex]


no image
Effects of Automated Skill Assessment on Robotic Surgery Training

Brown, J. D., Kuchenbecker, K. J.

The International Journal of Medical Robotics and Computer Assisted Surgery, 19(2):e2492, April 2023 (article)

Abstract
Background: Several automated skill-assessment approaches have been proposed for robotic surgery, but their utility is not well understood. This article investigates the effects of one machine-learning-based skill-assessment approach on psychomotor skill development in robotic surgery training. Methods: N=29 trainees (medical students and residents) with no robotic surgery experience performed five trials of inanimate peg transfer with an Intuitive Surgical da Vinci Standard robot. Half of the participants received no post-trial feedback. The other half received automatically calculated scores from five Global Evaluative Assessment of Robotic Skill (GEARS) domains post-trial. Results: There were no significant differences between the groups regarding overall improvement or skill improvement rate. However, participants who received post-trial feedback rated their overall performance improvement significantly lower than participants who did not receive feedback. Conclusions: These findings indicate that automated skill evaluation systems might improve trainee selfawareness but not accelerate early-stage psychomotor skill development in robotic surgery training.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


ForageFeeder: a low-cost open source feeder for randomly distributed food
ForageFeeder: a low-cost open source feeder for randomly distributed food

Jadali, N., Zhang, M., Schulz, A., Meyerchick, J., Schulz, A.

HardwareX, March 2023 (article)

Abstract
Automated feeders have long fed mice, livestock, and poultry, but are incapable of feeding zoo animals such as gorillas. In captivity, gorillas eat cut vegetables and fruits in pieces too large to be dispensed by automated feeders. Consequently, captive gorillas are fed manually at set times and locations, keeping them from the exercise and enrichment that accompanies natural foraging. We designed and built ForageFeeder, an automated gorilla feeder that spreads food at random intervals throughout the day. ForageFeeder is an open source and easy to manufacture and modify device, making the feeder more accessible for zoos. The design presented here reduces manual labor for zoo staff and may be a useful tool for studies of animal ethology.

DOI [BibTex]

DOI [BibTex]


no image
The S-BAN: Insights into the Perception of Shape-Changing Haptic Interfaces via Virtual Pedestrian Navigation

Spiers, A. J., Young, E., Kuchenbecker, K. J.

ACM Transactions on Computer-Human Interaction, 30(1):1-31, March 2023 (article)

Abstract
Screen-based pedestrian navigation assistance can be distracting or inaccessible to users. Shape-changing haptic interfaces can overcome these concerns. The S-BAN is a new handheld haptic interface that utilizes a parallel kinematic structure to deliver 2-DOF spatial information over a continuous workspace, with a form factor suited to integration with other travel aids. The ability to pivot, extend and retract its body opens possibilities and questions around spatial data representation. We present a static study to understand user perception of absolute pose and relative motion for two spatial mappings, showing highest sensitivity to relative motions in the cardinal directions. We then present an embodied navigation experiment in virtual reality. User motion efficiency when guided by the S-BAN was statistically equivalent to using a vision-based tool (a smartphone proxy). Although haptic trials were slower than visual trials, participants' heads were more elevated with the S-BAN, allowing greater visual focus on the environment.

DOI Project Page [BibTex]


Elephant trunks use an adaptable prehensile grip
Elephant trunks use an adaptable prehensile grip

Schulz, A., Reidenberg, J., Wu, J. N., Tang, C. Y., Seleb, B., Mancebo, J., Elgart, N., Hu, D.

Bioinspiration and Biomimetics, 18, February 2023 (article)

Abstract
Elephants have long been observed to grip objects with their trunk, but little is known about how they adjust their strategy for different weights. In this study, we challenge a female African elephant at Zoo Atlanta to lift 20–60 kg barbell weights with only its trunk. We measure the trunk’s shape and wrinkle geometry from a frozen elephant trunk at the Smithsonian. We observe several strategies employed to accommodate heavier weights, including accelerating less, orienting the trunk vertically, and wrapping the barbell with a greater trunk length. Mathematical models show that increasing barbell weights are associated with constant trunk tensile force and an increasing barbell-wrapping surface area due to the trunk’s wrinkles. Our findings may inspire the design of more adaptable soft robotic grippers that can improve grip using surface morphology such as wrinkles.

DOI [BibTex]

DOI [BibTex]


Drying dynamics of pellet feces
Drying dynamics of pellet feces

Magondu, B., Lee, A., Schulz, A., Buchelli, G., Meng, M., Kaminski, C., Yang, P., Carver, S., Hu, D.

Soft Matter, 19, pages: 723-732, January 2023 (article)

Abstract
Pellet feces are generated by a number of animals important to science or agriculture, including mice, rats, goats, and wombats. Understanding the factors that lead to fecal shape may provide a better understanding of animal health and diet. In this combined experimental and theoretical study, we test the hypothesis that pellet feces are formed by drying processes in the intestine. Inspirational to our work is the formation of hexagonal columnar jointings in cooling lava beds, in which the width L of the hexagon scales as L ∼ J−1 where J is the heat flux from the bed. Across 22 species of mammals, we report a transition from cylindrical to pellet feces if fecal water content drops below 0.65. Using a mathematical model that accounts for water intake rate and intestinal dimensions, we show pellet feces length L scales as L ∼ J−2.08 where J is the flux of water absorbed by the intestines. We build a mimic of the mammalian intestine using a corn starch cake drying in an open trough, finding that corn starch pellet length scales with water flux−0.46. The range of exponents does not permit us to conclude that formation of columnar jointings is similar to the formation of pellet feces. Nevertheless, the methods and physical picture shown here may be of use to physicians and veterinarians interested in using feces length as a marker of intestinal health.

DOI [BibTex]

DOI [BibTex]


The Utility of Synthetic Reflexes and Haptic Feedback for Upper-Limb Prostheses in a Dexterous Task Without Direct Vision
The Utility of Synthetic Reflexes and Haptic Feedback for Upper-Limb Prostheses in a Dexterous Task Without Direct Vision

Thomas, N., Fazlollahi, F., Kuchenbecker, K. J., Brown, J. D.

IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31, pages: 169-179, January 2023 (article)

Abstract
Individuals who use myoelectric upper-limb prostheses often rely heavily on vision to complete their daily activities. They thus struggle in situations where vision is overloaded, such as multitasking, or unavailable, such as poor lighting conditions. Able-bodied individuals can easily accomplish such tasks due to tactile reflexes and haptic sensation guiding their upper-limb motor coordination. Based on these principles, we developed and tested two novel prosthesis systems that incorporate autonomous controllers and provide the user with touch-location feedback through either vibration or distributed pressure. These capabilities were made possible by installing a custom contact-location sensor on the fingers of a commercial prosthetic hand, along with a custom pressure sensor on the thumb. We compared the performance of the two systems against a standard myoelectric prosthesis and a myoelectric prosthesis with only autonomous controllers in a difficult reach-to-pick-and-place task conducted without direct vision. Results from 40 able-bodied participants in this between-subjects study indicated that vibrotactile feedback combined with synthetic reflexes proved significantly more advantageous than the standard prosthesis in several of the task milestones. In addition, vibrotactile feedback and synthetic reflexes improved grasp placement compared to only synthetic reflexes or pressure feedback combined with synthetic reflexes. These results indicate that autonomous controllers and haptic feedback together facilitate success in dexterous tasks without vision, and that the type of haptic display matters.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Predicting the Force Map of an {ERT}-Based Tactile Sensor Using Simulation and Deep Networks
Predicting the Force Map of an ERT-Based Tactile Sensor Using Simulation and Deep Networks

Lee, H., Sun, H., Park, H., Serhat, G., Javot, B., Martius, G., Kuchenbecker, K. J.

IEEE Transactions on Automation Science and Engineering, 20(1):425-439, January 2023 (article)

Abstract
Electrical resistance tomography (ERT) can be used to create large-scale soft tactile sensors that are flexible and robust. Good performance requires a fast and accurate mapping from the sensor's sequential voltage measurements to the distribution of force across its surface. However, particularly with multiple contacts, this task is challenging for both previously developed approaches: physics-based modeling and end-to-end data-driven learning. Some promising results were recently achieved using sim-to-real transfer learning, but estimating multiple contact locations and accurate contact forces remains difficult because simulations tend to be less accurate with a high number of contact locations and/or high force. This paper introduces a modular hybrid method that combines simulation data synthesized from an electromechanical finite element model with real measurements collected from a new ERT-based tactile sensor. We use about 290,000 simulated and 90,000 real measurements to train two deep neural networks: the first (Transfer-Net) captures the inevitable gap between simulation and reality, and the second (Recon-Net) reconstructs contact forces from voltage measurements. The number of contacts, contact locations, force magnitudes, and contact diameters are evaluated for a manually collected multi-contact dataset of 150 measurements. Our modular pipeline's results outperform predictions by both a physics-based model and end-to-end learning.

DOI Project Page [BibTex]


Bioinspired Robots Can Foster Nature Conservation
Bioinspired Robots Can Foster Nature Conservation

Chellapurath, M., Khandelwal, P., Schulz, A. K.

Frontiers in Robotics and AI, 10, 2023 (article) Accepted

Abstract
We live in a time of unprecedented scientific and human progress while being increasingly aware of its negative impacts on our planet's health. Aerial, terrestrial, and aquatic ecosystems have significantly declined putting us on course to a sixth mass extinction event. Nonetheless, the advances made in science, engineering, and technology have given us the opportunity to reverse some of our ecosystem damage and preserve them through conservation efforts around the world. However, current conservation efforts are primarily human led with assistance from conventional robotic systems which limit their scope and effectiveness, along with negatively impacting the surroundings. In this perspective, we present the field of bioinspired robotics to develop versatile agents for future conservation efforts that can operate in the natural environment while minimizing the disturbance/impact to its inhabitants and the environment's natural state. We provide an operational and environmental framework that should be considered while developing bioinspired robots for conservation. These considerations go beyond addressing the challenges of human-led conservation efforts and leverage the advancements in the field of materials, intelligence, and energy harvesting, to make bioinspired robots move and sense like animals. In doing so, it makes bioinspired robots an attractive, non-invasive, sustainable, and effective conservation tool for exploration, data collection, intervention, and maintenance. Finally, we discuss the development of bioinspired robots in the context of collaboration, practicality, and applicability that would ensure their further development and widespread use to protect and preserve our natural world.

[BibTex]

[BibTex]


Conservation Tools: The Next Generation of Engineering--Biology Collaborations
Conservation Tools: The Next Generation of Engineering–Biology Collaborations

Schulz, A., Shriver, C., Stathatos, S., Seleb, B., Weigel, E., Chang, Y., Bhamla, M. S., Hu, D., III, J. R. M.

Royal Society Interface, 2023, Andrew Schulz, Cassie Shriver, Suzanne Stathatos, and Benjamin Seleb are co-first authors. (article)

Abstract
The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in wildlife conservation. In this review, we present five case studies and infer a framework for designing conservation tools based on human-wildlife interaction. Successful conservation tools range in complexity from cat collars to machine learning and game theory methodologies and do not require technological expertise to contribute to conservation tool creation. We aim to introduce researchers to conservation technology and provide references for guiding the next generation of conservation technologists. Conservation technology has the potential to benefit biodiversity and have broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.

DOI [BibTex]

DOI [BibTex]


A Year at the Forefront of Hydrostat Motion
A Year at the Forefront of Hydrostat Motion

Schulz, A., Schneider, N., Zhang, M., Singal, K.

Biology Open, 2023, N. Schneider, M. Zhang, and K. Singal all contributed equally on this manuscript. (article)

Abstract
Currently, in the field of interdisciplinary work in biology, there has been a significant push by the soft robotic community to understand the motion and maneuverability of hydrostats. This review seeks to expand the muscular hydrostat hypothesis toward new structures, including plants, and introduce innovative techniques to the hydrostat community on new modeling, simulating, mimicking, and observing hydrostat motion methods. These methods range from ideas of kirigami, origami, and knitting for mimic creation to utilizing reinforcement learning for control of bio-inspired soft robotic systems. It is now being understood through modeling that different mechanisms can inhibit traditional hydrostat motion, such as skin, nostrils, or sheathed layered muscle walls. The impact of this review will highlight these mechanisms, including asymmetries, and discuss the critical next steps toward understanding their motion and how species with hydrostat structures control such complex motions, highlighting work from January 2022 to December 2022.

DOI [BibTex]

DOI [BibTex]

2022


A Guide for Successful Research Collaborations between Zoos and Universities
A Guide for Successful Research Collaborations between Zoos and Universities

Schulz, A., Shriver, C., Aubuchon, C., Weigel, E., Kolar, M., III, J. M., Hu, D.

Integrative and Comparitive Biology, 62, pages: 1174-1185, November 2022 (article)

Abstract
Zoos offer university researchers unique opportunities to study animals that would be difficult or impractical to work with in the wild. However, the different cultures, goals, and priorities of zoos and universities can be a source of conflict. How can researchers build mutually beneficial collaborations with their local zoo? In this article, we present the results of a survey of 117 personnel from 59 zoos around the United States, where we highlight best practices spanning all phases of collaboration, from planning to working alongside the zoo and maintaining contact afterward. Collaborations were hindered if university personnel did not appreciate the zoo staff’s time constraints as well as the differences between zoo animals and laboratory animals. We include a vision for how to improve zoo collaborations, along with a history of our own decade-long collaborations with Zoo Atlanta. A central theme is the long-term establishment of trust between institutions.

DOI [BibTex]

2022

DOI [BibTex]


no image
Learning to Feel Textures: Predicting Perceptual Similarities from Unconstrained Finger-Surface Interactions

Richardson, B. A., Vardar, Y., Wallraven, C., Kuchenbecker, K. J.

IEEE Transactions on Haptics, 15(4):705-717, October 2022, Benjamin A. Richardson and Yasemin Vardar contributed equally to this publication. (article)

Abstract
Whenever we touch a surface with our fingers, we perceive distinct tactile properties that are based on the underlying dynamics of the interaction. However, little is known about how the brain aggregates the sensory information from these dynamics to form abstract representations of textures. Earlier studies in surface perception all used general surface descriptors measured in controlled conditions instead of considering the unique dynamics of specific interactions, reducing the comprehensiveness and interpretability of the results. Here, we present an interpretable modeling method that predicts the perceptual similarity of surfaces by comparing probability distributions of features calculated from short time windows of specific physical signals (finger motion, contact force, fingernail acceleration) elicited during unconstrained finger-surface interactions. The results show that our method can predict the similarity judgments of individual participants with a maximum Spearman's correlation of 0.7. Furthermore, we found evidence that different participants weight interaction features differently when judging surface similarity. Our findings provide new perspectives on human texture perception during active touch, and our approach could benefit haptic surface assessment, robotic tactile perception, and haptic rendering.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Skin wrinkles and folds enable asymmetric stretch in the elephant trunk
Skin wrinkles and folds enable asymmetric stretch in the elephant trunk

Schulz, A., Boyle, M., Boyle, C., Sordilla, S., Rincon, C., Hooper, S., Aubuchon, C., Reidenberg, J., Higgins, C., Hu, D.

Proceedings of the National Academices of Science , 119, August 2022 (article)

Abstract
The elephant’s trunk is multifunctional: It must be flexible to wrap around vegetation, but tough to knock down trees and resist attack. How can one appendage satisfy both constraints? In this combined experimental and theoretical study, we challenged African elephants to reach far-away objects with only horizontal extensions of their trunk. Surprisingly, the trunk does not extend uniformly, but instead exhibits a dorsal “joint” that stretches 15% more than the corresponding ventral section. Using material testing with the skin of a deceased elephant, we show that the asymmetry is due in part to patterns of the skin. The dorsal skin is folded and 15% more pliable than the wrinkled ventral skin. Skin folds protect the dorsal section and stretch to facilitate downward wrapping, the most common gripping style when picking up items. The elephant’s skin is also sufficiently stiff to influence its mechanics: At the joint, the skin requires 13 times more energy to stretch than the corresponding length of muscle. The use of wrinkles and folds to modulate stiffness may provide a valuable concept for both biology and soft robotics.

DOI [BibTex]

DOI [BibTex]


no image
Maximizing buckling load of elliptical composite cylinders using lamination parameters

Guo, Y., Serhat, G., Pérez, M. G., Knippers, J.

Engineering Structures, 262, pages: 114342, July 2022 (article)

Abstract
Structural members made of fiber-reinforced polymers (FRP) attract increasing attention in the development of novel architectural systems that challenge the standard design methodologies. Cylindrical surfaces constitute one of the typical geometric sets obtained with the FRP component fabrication. This paper explores the influences of two geometric parameters on the buckling performance of elliptical cylinders: inverse slenderness (ratio of minimum diameter to height) and eccentricity (ratio of radii along semi-axes). The overall stiffness properties are defined using lamination parameters. This analysis method eliminates the dependency of optimal solutions on the initial assumptions regarding the laminate configuration, which needs to be explicitly described in multi-layer modeling. Finite element analyses are utilized to compute buckling loads of the cylinders under axial compression force and bi-axial bending moments. The optimal lamination parameters and buckling stresses are determined for various parameters, and the lamination parameters corresponding to the optimal and simple [±45°] angle-ply design points are presented in the lamination parameter plane via Miki's diagram. The results reveal the level of performance that can be achieved by a specific geometry and provide guidelines for the optimal design of elliptical laminated cylinders against buckling.

DOI [BibTex]

DOI [BibTex]


no image
The Scaling of Olfaction: Moths have Relatively More Olfactory Surface Area than Mammals

Mohebbi, N., Schulz, A., Spencer, T., Pos, K., Mandel, A., Casas, J., Hu, D.

Integrative and Comparitive Biology, 62(1), July 2022, Nina Mohebbi and Andrew Schulz contributed equally. (article)

Abstract
Body size affects nearly every aspect of locomotion and sensing, but little is known of its influence on olfaction. One reason for this missing link is that olfaction differs fundamentally from vision and hearing in that molecules are advected by fluid before depositing on olfactory sensors. This critical role of fluid flow in olfaction leads to complexities and trade-offs. For example, a greater density of hairs and sensory neurons may lead to greater collection, but can also lead to reduced flow through hairs and additional weight and drag due to a larger olfactory organ. In this study, we report the surface area and sensory neuron density in olfactory organs of 95 species of moths and mammals. We find that approximately 12–14% of an olfactory system’s surface area is devoted to chemosensors. Furthermore, total olfactory surface area and olfactory sensing surface area scale with body mass to the 0.49 and 0.38 powers, respectively, indicating that moths have a higher proportion of olfactory surface area than mammals. The density of olfactory neurons appears to be near the limit, at 10,000 to 100,000 neurons per square mm across both insects and mammals. This study demonstrates the need for future work detailing how the scaling of olfaction and other senses vary across taxa.

DOI [BibTex]

DOI [BibTex]


no image
Contact Evolution of Dry and Hydrated Fingertips at Initial Touch

Serhat, G., Vardar, Y., Kuchenbecker, K. J.

PLOS ONE, 17(7):e0269722, July 2022, Gokhan Serhat and Yasemin Vardar contributed equally to this publication. (article)

Abstract
Pressing the fingertips into surfaces causes skin deformations that enable humans to grip objects and sense their physical properties. This process involves intricate finger geometry, non-uniform tissue properties, and moisture, complicating the underlying contact mechanics. Here we explore the initial contact evolution of dry and hydrated fingers to isolate the roles of governing physical factors. Two participants gradually pressed an index finger on a glass surface under three moisture conditions: dry, water-hydrated, and glycerin-hydrated. Gross and real contact area were optically measured over time, revealing that glycerin hydration produced strikingly higher real contact area, while gross contact area was similar for all conditions. To elucidate the causes for this phenomenon, we investigated the combined effects of tissue elasticity, skin-surface friction, and fingerprint ridges on contact area using simulation. Our analyses show the dominant influence of elastic modulus over friction and an unusual contact phenomenon, which we call friction-induced hinging.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Perceptual Space of Algorithms for Three-to-One Dimensional Reduction of Realistic Vibrations
Perceptual Space of Algorithms for Three-to-One Dimensional Reduction of Realistic Vibrations

Lee, H., Tombak, G. I., Park, G., Kuchenbecker, K. J.

IEEE Transactions on Haptics, 15(3):521-534, July 2022 (article)

Abstract
Haptics researchers often endeavor to deliver realistic vibrotactile feedback through broad-bandwidth actuators; however, these actuators typically generate only single-axis vibrations, not 3D vibrations like those that occur in natural tool-mediated interactions. Several three-to-one (321) dimensional reduction algorithms have thus been developed to combine 3D vibrations into 1D vibrations. Surprisingly, the perceptual quality of 321-converted vibrations has never been comprehensively compared to rendering of the original 3D signals. In this study, we develop a multi-dimensional vibration rendering system using a magnetic levitation haptic interface. We verify the system's ability to generate realistic 3D vibrations recorded in both tapping and dragging interactions with four surfaces. We then conduct a study with 15 participants to measure the perceived dissimilarities between five 321 algorithms (SAZ, SUM, VM, DFT, PCA) and the original recordings. The resulting perceptual space is investigated with multiple regression and Procrustes analysis to unveil the relationship between the physical and perceptual properties of 321-converted vibrations. Surprisingly, we found that participants perceptually discriminated the original 3D vibrations from all tested 1D versions. Overall, our results indicate that spectral, temporal, and directional attributes may all contribute to the perceived similarities of vibration signals.

DOI Project Page [BibTex]


no image
Multi-objective optimization of composite sandwich panels using lamination parameters and spectral Chebyshev method

Seyyedrahmani, F., Shahabad, P. K., Serhat, G., Bediz, B., Basdogan, I.

Composite Structures, 289(115417), June 2022 (article)

Abstract
Composite materials are widely used in various industries because of their distinct properties. Hybridization is an efficient way of designing composite panels to decrease the cost and/or weight while maintaining stiffness properties. In this study, an accurate and efficient framework is developed to optimize laminated sandwich panels composed of high-stiffness face sheets and low-stiffness core. The stiffness properties of face sheets and core are represented using lamination parameters. The governing equations are derived following first-order shear deformation theory and solved using the spectral Chebyshev approach. In multi-objective optimization problems, genetic algorithm is used to determine Pareto-optimal solutions for fundamental frequency, frequency gap, buckling load, and cost metrics. In these analyses, optimal lamination parameters and thickness are found for face-sheets and core of sandwich panels, and the results are presented as 2D and 3D Pareto-optimal design points. When the individual performance metrics lead to different optimum points, a scattering behavior is observed in the 3D Pareto sets whose boundaries are defined by the 2-objective Pareto fronts. The results provide insights into the design requirements for improving the dynamic and load-carrying behavior of sandwich laminates while minimizing the cost that presents the usability of the presented approach in the multi-objective optimization.

DOI [BibTex]

DOI [BibTex]


Normal and Tangential Forces Combine to Convey Contact Pressure During Dynamic Tactile Stimulation
Normal and Tangential Forces Combine to Convey Contact Pressure During Dynamic Tactile Stimulation

Gueorguiev, D., Lambert, J., Thonnard, J., Kuchenbecker, K. J.

Scientific Reports, 12, pages: 8215, May 2022 (article)

Abstract
Humans need to accurately process the contact forces that arise as they perform everyday haptic interactions such as sliding the fingers along a surface to feel for bumps, sticky regions, or other irregularities. Several different mechanisms are possible for how the forces on the skin could be represented and integrated in such interactions. In this study, we used a force-controlled robotic platform and simultaneous ultrasonic modulation of the finger-surface friction to independently manipulate the normal and tangential forces during passive haptic stimulation by a flat surface. To assess whether the contact pressure on their finger had briefly increased or decreased during individual trials in this broad stimulus set, participants did not rely solely on either the normal force or the tangential force. Instead, they integrated tactile cues induced by both components. Support-vector-machine analysis classified physical trial data with up to 75% accuracy and suggested a linear perceptual mechanism. In addition, the change in the amplitude of the force vector predicted participants' responses better than the change of the coefficient of dynamic friction, suggesting that intensive tactile cues are meaningful in this task. These results provide novel insights about how normal and tangential forces shape the perception of tactile contact.

DOI Project Page [BibTex]


no image
Predicting Knee Adduction Moment Response to Gait Retraining with Minimal Clinical Data

Rokhmanova, N., Kuchenbecker, K. J., Shull, P. B., Ferber, R., Halilaj, E.

PLOS Computational Biology, 18(5):e1009500, May 2022 (article)

Abstract
Knee osteoarthritis is a progressive disease mediated by high joint loads. Foot progression angle modifications that reduce the knee adduction moment (KAM), a surrogate of knee loading, have demonstrated efficacy in alleviating pain and improving function. Although changes to the foot progression angle are overall beneficial, KAM reductions are not consistent across patients. Moreover, customized interventions are time-consuming and require instrumentation not commonly available in the clinic. We present a regression model that uses minimal clinical data-a set of six features easily obtained in the clinic-to predict the extent of first peak KAM reduction after toe-in gait retraining. For such a model to generalize, the training data must be large and variable. Given the lack of large public datasets that contain different gaits for the same patient, we generated this dataset synthetically. Insights learned from a ground-truth dataset with both baseline and toe-in gait trials (N = 12) enabled the creation of a large (N = 138) synthetic dataset for training the predictive model. On a test set of data collected by a separate research group (N = 15), the first peak KAM reduction was predicted with a mean absolute error of 0.134% body weight * height (%BW*HT). This error is smaller than the standard deviation of the first peak KAM during baseline walking averaged across test subjects (0.306%BW*HT). This work demonstrates the feasibility of training predictive models with synthetic data and provides clinicians with a new tool to predict the outcome of patient-specific gait retraining without requiring gait lab instrumentation.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Design of Interactive Augmented Reality Functions for Robotic Surgery and Evaluation in Dry-Lab Lymphadenectomy
Design of Interactive Augmented Reality Functions for Robotic Surgery and Evaluation in Dry-Lab Lymphadenectomy

Forte, M., Gourishetti, R., Javot, B., Engler, T., Gomez, E. D., Kuchenbecker, K. J.

The International Journal of Medical Robotics and Computer Assisted Surgery, 18(2):e2351, April 2022 (article)

Abstract
Augmented reality (AR) has been widely researched for use in healthcare. Prior AR for robot-assisted minimally invasive surgery has mainly focused on superimposing preoperative 3D images onto patient anatomy. This paper presents alternative interactive AR tools for robotic surgery. We designed, built, and evaluated four voice-controlled functions: viewing a live video of the operating room, viewing two-dimensional preoperative images, measuring 3D distances, and warning about out-of-view instruments. This low-cost system was developed on a da Vinci Si, and it can be integrated into surgical robots equipped with a stereo camera and a stereo viewer. Eight experienced surgeons performed dry-lab lymphadenectomies and reported that the functions improved the procedure. They particularly appreciated the possibility of accessing the patient's medical records on demand, measuring distances intraoperatively, and interacting with the functions using voice commands. The positive evaluations garnered by these alternative AR functions and interaction methods provide support for further exploration.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


Endowing a {NAO} Robot with Practical Social-Touch Perception
Endowing a NAO Robot with Practical Social-Touch Perception

Burns, R. B., Lee, H., Seifi, H., Faulkner, R., Kuchenbecker, K. J.

Frontiers in Robotics and AI, 9, pages: 840335, April 2022 (article)

Abstract
Social touch is essential to everyday interactions, but current socially assistive robots have limited touch-perception capabilities. Rather than build entirely new robotic systems, we propose to augment existing rigid-bodied robots with an external touch-perception system. This practical approach can enable researchers and caregivers to continue to use robotic technology they have already purchased and learned about, but with a myriad of new social-touch interactions possible. This paper presents a low-cost, easy-to-build, soft tactile-perception system that we created for the NAO robot, as well as participants' feedback on touching this system. We installed four of our fabric-and-foam-based resistive sensors on the curved surfaces of a NAO's left arm, including its hand, lower arm, upper arm, and shoulder. Fifteen adults then performed five types of affective touch-communication gestures (hitting, poking, squeezing, stroking, and tickling) at two force intensities (gentle and energetic) on the four sensor locations; we share this dataset of four time-varying resistances, our sensor patterns, and a characterization of the sensors' physical performance. After training, a gesture-classification algorithm based on a random forest identified the correct combined touch gesture and force intensity on windows of held-out test data with an average accuracy of 74.1%, which is more than eight times better than chance. Participants rated the sensor-equipped arm as pleasant to touch and liked the robot's presence significantly more after touch interactions. Our promising results show that this type of tactile-perception system can detect necessary social-touch communication cues from users, can be tailored to a variety of robot body parts, and can provide HRI researchers with the tools needed to implement social touch in their own systems.

DOI Project Page [BibTex]


no image
Adaptive Optimal Measurement Algorithm for ERT-Based Large-Area Tactile Sensors

Park, K., Lee, H., Kuchenbecker, K. J., Kim, J.

IEEE/ASME Transactions on Mechatronics, 27(1):304-314, February 2022 (article)

Abstract
Electrical resistance tomography (ERT) is an inferential imaging technique that has shown promising results for enabling large-area tactile sensors constructed from a piezoresistive sheet. The performance of such sensors is improved by increasing the number of electrodes, but the number of measurements and the computational cost also increase. In this article, we propose a new measurement algorithm for ERT-based tactile sensors: it adaptively changes the measurement pattern to be optimal for the present external stimulus. Regions of normal pressure are first detected by a base measurement pattern that maximizes the distinguishability of local conductivity changes. When a new contact is detected, a set of local patterns is selectively recruited near the pressed region to acquire more detailed information. For fast and parallel execution, the proposed algorithm is implemented with a field-programmable gate array. It is validated through indentation experiments on an ERT-based sensor that has 32 electrodes. The optimized base pattern of 100 measurements enabled a frame rate five times faster than before. Transmitting only detected contact events reduced the idle data rate to 0.5% of its original value. The pattern adapted to new contacts with a latency of only 80 μs and an accuracy of 99.5%, enabling efficient, high-quality real-time reconstruction of complex multicontact conditions.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
A design methodology for fiber layup optimization of filament wound structural components

Guo, Y., Pérez, M. G., Serhat, G., Knippers, J.

Structures, 38, pages: 1125-1136, February 2022 (article)

Abstract
The applications of fiber-reinforced polymer (FRP) composites extend rapidly along with the development of new manufacturing techniques. However, due to the complexities introduced by the material and fabrication processes, the application of conventional structural design methods for construction members has been significantly challenging. This paper presents an alternative methodology to find optimum fiber layups for a given tube-shape geometry via a graphical optimization strategy based on structural performance requirements. The proposed technique employs simplified shell element models based on classical lamination theory (CLT) to avoid explicit fiber modeling in the FEA simulations. Lamination parameters are utilized to generate the reduced stiffness matrices for continuous multi-layer FRP lamination. The fiber layup of the component is retrieved from the optimal lamination parameters that maximize the structural performance. The case study results demonstrate that the developed method provides compact solutions, linking the structural design requirements with optimal fiber orientations and volumetric proportions. In addition, the determined solutions can be interpreted directly by the winding fabrication settings.

DOI [BibTex]

DOI [BibTex]


A Soft Thumb-Sized Vision-Based Sensor with Accurate All-Round Force Perception
A Soft Thumb-Sized Vision-Based Sensor with Accurate All-Round Force Perception

Sun, H., Kuchenbecker, K. J., Martius, G.

Nature Machine Intelligence, 4(2):135-145, February 2022 (article)

Abstract
Vision-based haptic sensors have emerged as a promising approach to robotic touch due to affordable high-resolution cameras and successful computer-vision techniques. However, their physical design and the information they provide do not yet meet the requirements of real applications. We present a robust, soft, low-cost, vision-based, thumb-sized 3D haptic sensor named Insight: it continually provides a directional force-distribution map over its entire conical sensing surface. Constructed around an internal monocular camera, the sensor has only a single layer of elastomer over-molded on a stiff frame to guarantee sensitivity, robustness, and soft contact. Furthermore, Insight is the first system to combine photometric stereo and structured light using a collimator to detect the 3D deformation of its easily replaceable flexible outer shell. The force information is inferred by a deep neural network that maps images to the spatial distribution of 3D contact force (normal and shear). Insight has an overall spatial resolution of 0.4 mm, force magnitude accuracy around 0.03 N, and force direction accuracy around 5 degrees over a range of 0.03--2 N for numerous distinct contacts with varying contact area. The presented hardware and software design concepts can be transferred to a wide variety of robot parts.

Paper link (url) DOI Project Page [BibTex]


no image
Lessons Learned About Designing and Conducting Studies From HRI Experts

Fraune, M. R., Leite, I., Karatas, N., Amirova, A., Legeleux, A., Sandygulova, A., Neerincx, A., Tikas, G. D., Gunes, H., Mohan, M., Abbasi, N. I., Shenoy, S., Scassellati, B., Visser, E. J. D., Komatsu, T.

Frontiers in Robotics and AI, 8(772141):1-18, January 2022 (article)

Abstract
The field of human-robot interaction (HRI) research is multidisciplinary and requires researchers to understand diverse fields including computer science, engineering, informatics, philosophy, psychology, and more disciplines. However, it is hard to be an expert in everything. To help HRI researchers develop methodological skills, especially in areas that are relatively new to them, we conducted a virtual workshop, Workshop Your Study Design (WYSD), at the 2021 International Conference on HRI. In this workshop, we grouped participants with mentors, who are experts in areas like real-world studies, empirical lab studies, questionnaire design, interview, participatory design, and statistics. During and after the workshop, participants discussed their proposed study methods, obtained feedback, and improved their work accordingly. In this paper, we present 1) Workshop attendees' feedback about the workshop and 2) Lessons that the participants learned during their discussions with mentors. Participants' responses about the workshop were positive, and future scholars who wish to run such a workshop can consider implementing their suggestions. The main contribution of this paper is the lessons learned section, where the workshop participants contributed to forming this section based on what participants discovered during the workshop. We organize lessons learned into themes of 1) Improving study design for HRI, 2) How to work with participants - especially children -, 3) Making the most of the study and robot's limitations, and 4) How to collaborate well across fields as they were the areas of the papers submitted to the workshop. These themes include practical tips and guidelines to assist researchers to learn about fields of HRI research with which they have limited experience. We include specific examples, and researchers can adapt the tips and guidelines to their own areas to avoid some common mistakes and pitfalls in their research.

DOI [BibTex]

DOI [BibTex]


no image
Evaluation of Vibrotactile Output from a Rotating Motor Actuator

(Honorable Mention for the Best ToH Short Paper Award at the IEEE Haptics Symposium Conference 2022)

Gourishetti, R., Kuchenbecker, K. J.

IEEE Transactions on Haptics, 15(1):39-44, January 2022, Presented at the IEEE Haptics Symposium (article)

Abstract
Specialized vibrotactile actuators are widely used to output haptic sensations due to their portability and robustness; some models are expensive and capable, while others are economical but weaker and less expressive. To increase the accessibility of high-quality haptics, we designed a cost-effective actuation approach called the rotating motor actuator (RMA): it uses a small DC motor to generate vibrotactile cues on a rigid stylus. We conducted a psychophysical experiment where eighteen volunteers matched the RMA's vibration amplitudes with those from a high-quality reference actuator (Haptuator Mark II) at twelve frequencies from 50 Hz to 450 Hz. The average error in matching acceleration magnitudes was 10.2%. More current was required for the RMA than the reference actuator; a stronger DC motor would require less current. Participants also watched a video of a real tool-mediated interaction with playback of recorded vibrotactile cues from each actuator. 94.4% of the participants agreed that the RMA delivered realistic vibrations and audio cues during this replay. 83.3% reported that the RMA vibrations were pleasant, compared to 66.7% for the reference. A possible cause for this significant difference may be that the reference actuator (which has a mechanical resonance) distorts low-frequency vibrations more than the RMA does.

DOI Project Page [BibTex]

DOI Project Page [BibTex]

2021


Virtual Reality Treatment Displaying the Missing Leg Improves Phantom Limb Pain: A Small Clinical Trial
Virtual Reality Treatment Displaying the Missing Leg Improves Phantom Limb Pain: A Small Clinical Trial

Ambron, E., Buxbaum, L. J., Miller, A., Stoll, H., Kuchenbecker, K. J., Coslett, H. B.

Neurorehabilitation and Neural Repair, 35(12):1100-1111, December 2021 (article)

Abstract
Background: Phantom limb pain (PLP) is a common and in some cases debilitating consequence of upper- or lower-limb amputation for which current treatments are inadequate. Objective: This small clinical trial tested whether game-like interactions with immersive VR activities can reduce PLP in subjects with transtibial lower-limb amputation. Methods: Seven participants attended 5–7 sessions in which they engaged in a visually immersive virtual reality experience that did not require leg movements (Cool! TM), followed by 10–12 sessions of targeted lower-limb VR treatment consisting of custom games requiring leg movement. In the latter condition, they controlled an avatar with 2 intact legs viewed in a head-mounted display (HTC Vive TM). A motion-tracking system mounted on the intact and residual limbs controlled the movements of both virtual extremities independently. Results: All participants except one experienced a reduction of pain immediately after VR sessions, and their pre session pain levels also decreased over the course of the study. At a group level, PLP decreased by 28% after the treatment that did not include leg movements and 39.6% after the games requiring leg motions. Both treatments were successful in reducing PLP. Conclusions: This VR intervention appears to be an efficacious treatment for PLP in subjects with lower-limb amputation.

DOI Project Page [BibTex]

2021

DOI Project Page [BibTex]


no image
Object Handovers: A Review for Robotics

Ortenzi, V., Cosgun, A., Pardi, T., Chan, W. P., Croft, E., Kulić, D.

IEEE Transactions on Robotics, 37(6):1855-1873, December 2021 (article)

Abstract
This article surveys the literature on human–robot object handovers. A handover is a collaborative joint action, where an agent, the giver, gives an object to another agent, the receiver. The physical exchange starts when the receiver first contacts the object held by the giver and ends when the giver fully releases the object to the receiver. However, important cognitive and physical processes begin before the physical exchange, including initiating implicit agreement with respect to the location and timing of the exchange. From this perspective, we structure our review into the two main phases delimited by the aforementioned events: a prehandover phase and the physical exchange. We focus our analysis on the two actors (giver and receiver) and report the state of the art of robotic givers (robot-to-human handovers) and the robotic receivers (human-to-robot handovers). We report a comprehensive list of qualitative and quantitative metrics commonly used to assess the interaction. While focusing our review on the cognitive level (e.g., prediction, perception, motion planning, and learning) and the physical level (e.g., motion, grasping, and grip release) of the handover, we also discuss safety. We compare the behaviors displayed during human-to-human handovers to the state of the art of robotic assistants and identify the major areas of improvement for robotic assistants to reach performance comparable to human interactions. Finally, we propose a minimal set of metrics that should be used in order to enable a fair comparison among the approaches.

DOI [BibTex]

DOI [BibTex]


no image
Shared Steering Control: How Strong and How Prompt Should the Intervention Be for a Better Driving Experience?

Park, K., Han, S. H., Lee, H., Kwahk, J.

International Journal of Industrial Ergonomics, 86(103213):1-9, Elsevier, November 2021 (article)

Abstract
The lane keeping assistance system, a representative advanced driver assistance system, comprises a shared control that cooperates with the driver to achieve a common goal. The steering experience of the driver may vary significantly depending on the auto-steering control strategy of the system. In this study, we examined the driving experience with various steering control strategies. Nine control strategies (three torque amounts × three deviations in starting control) were established as prototypes. Eighteen drivers participated in the evaluation of each strategy in a highway environment on a driving simulator. A two-way repeated measure ANOVA was used to assess the effects of the system. Both the objective measures (standard deviation of lane position, steering reversal rate, and root mean square of lateral speed) and subjective measures (pleasure and arousal of emotion, trust, disturbance, and satisfaction) were evaluated and analyzed. The results showed that a torque amount of 3 Nm evoked feelings of high disturbance and negative emotional responses. A deviation in starting control (DEV) of 0.80 m yielded unstable lane keeping performances and evoked negative effects on pleasure, trust, and satisfaction. A regression model for the driver satisfaction recommended a torque of 2.32 Nm and a DEV of 0.27 m as the optimal design parameters. This proposed strategy is expected to improve the experience design of lateral semi-autonomous vehicles.

DOI [BibTex]

DOI [BibTex]


no image
Automating Endoscope Motion in Robotic Surgery: A Usability Study on da Vinci-Assisted Ex Vivo Neobladder Reconstruction

Col, T. D., Caccianiga, G., Catellani, M., Mariani, A., Ferro, M., Cordima, G., Momi, E. D., Ferrigno, G., Cobelli, O. D.

Frontiers in Robotics and AI, 8(707704):1-10, November 2021 (article)

Abstract
Robots for minimally invasive surgery introduce many advantages, but still require the surgeon to alternatively control the surgical instruments and the endoscope. This work aims at providing autonomous navigation of the endoscope during a surgical procedure. The autonomous endoscope motion was based on kinematic tracking of the surgical instruments and integrated with the da Vinci Research Kit. A preclinical usability study was conducted by 10 urologists. They carried out an ex vivo orthotopic neobladder reconstruction twice, using both traditional and autonomous endoscope control. The usability of the system was tested by asking participants to fill standard system usability scales. Moreover, the effectiveness of the method was assessed by analyzing the total procedure time and the time spent with the instruments out of the field of view. The average system usability score overcame the threshold usually identified as the limit to assess good usability (average score = 73.25 > 68). The average total procedure time with the autonomous endoscope navigation was comparable with the classic control (p = 0.85 > 0.05), yet it significantly reduced the time out of the field of view (p = 0.022 < 0.05). Based on our findings, the autonomous endoscope improves the usability of the surgical system, and it has the potential to be an additional and customizable tool for the surgeon that can always take control of the endoscope or leave it to move autonomously.

DOI [BibTex]

DOI [BibTex]


Robotics for Occupational Therapy: Learning Upper-Limb Exercises From Demonstrations
Robotics for Occupational Therapy: Learning Upper-Limb Exercises From Demonstrations

Hu, S., Mendonca, R., Johnson, M. J., Kuchenbecker, K. J.

IEEE Robotics and Automation Letters, 6(4):7781-7788, October 2021 (article)

Abstract
We describe a learning-from-demonstration technique that enables a general-purpose humanoid robot to lead a user through object-mediated upper-limb exercises. It needs only tens of seconds of training data from a therapist teleoperating the robot to do the task with the user. We model the robot behavior as a regression problem, inferring the desired robot effort using the end-effector's state (position and velocity). Compared to the conventional approach of learning time-based trajectories, our strategy produces customized robot behavior and eliminates the need to tune gains to adapt to the user's motor ability. In our study, one occupational therapist and six people with stroke trained a Willow Garage PR2 on three example tasks (periodic 1D and 2D motions plus episodic pick-and-place). They then repeatedly did the tasks with the robot and blindly compared the state- and time-based controllers learned from the training data. Our results show that working models were reliably obtained to allow the robot to do the exercise with the user; that our state-based approach enabled users to be more actively involved, allowed larger excursion, and generated power outputs more similar to the therapist demonstrations; and that the therapist found our strategy more agreeable than the traditional time-based approach.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


A Brake-Based Overground Gait Rehabilitation Device for Altering Propulsion Impulse Symmetry
A Brake-Based Overground Gait Rehabilitation Device for Altering Propulsion Impulse Symmetry

Hu, S., Fjeld, K., Vasudevan, E. V., Kuchenbecker, K. J.

Sensors, 21(19):6617, October 2021 (article)

Abstract
This paper introduces a new device for gait rehabilitation, the gait propulsion trainer (GPT). It consists of two main components (a stationary device and a wearable system) that work together to apply periodic stance-phase resistance as the user walks overground. The stationary device provides the resistance forces via a cable that tethers the user's pelvis to a magnetic-particle brake. The wearable system detects gait events via foot switches to control the timing of the resistance forces. A hardware verification test confirmed that the GPT functions as intended. We conducted a pilot study in which one healthy adult and one stroke survivor walked with the GPT with increasing resistance levels. As hypothesized, the periodic stance-phase resistance caused the healthy participant to walk asymmetrically, with greatly reduced propulsion impulse symmetry; as GPT resistance increased, the walking speed also decreased, and the propulsion impulse appeared to increase for both legs. In contrast, the stroke participant responded to GPT resistance by walking faster and more symmetrically in terms of both propulsion impulse and step length. Thus, this paper shows promising results of short-term training with the GPT, and more studies will follow to explore its long-term effects on hemiparetic gait.

DOI Project Page [BibTex]


no image
Highly contrastive, real-time modulation of light intensity by reversible stress-whitening of spontaneously formed nanocomposites: Application to wearable strain sensors

Park, S. K., Yun, S., Hwang, G., Choi, M., Kim, D. W., Lee, J., Park, B. J., Nam, S., Mun, H., Mun, S., Lim, J. M., Shin, E. J., Kyung, K., Park, S.

Journal of Materials Chemistry C, 9(27):8496-8505, July 2021 (article)

link (url) DOI [BibTex]

link (url) DOI [BibTex]


Piezoresistive Textile Layer and Distributed Electrode Structure for Soft Whole-Body Tactile Skin
Piezoresistive Textile Layer and Distributed Electrode Structure for Soft Whole-Body Tactile Skin

Lee, H., Park, K., Kim, J., Kuchenbecker, K. J.

Smart Materials and Structures, 30(8):085036, July 2021, Hyosang Lee and Kyungseo Park contributed equally to this publication. (article)

Abstract
Tactile sensors based on electrical resistance tomography (ERT) provide pressure sensing over a large area using only a few electrodes, which is a promising property for robotic tactile skin. Most ERT-based tactile sensors employ electrodes only on the sensor's edge to avoid undesirable artifacts caused by electrode contact. The distribution of these electrodes is critical, as electrode location largely determines the sensitive regions, but only a few studies have positioned electrodes in the sensor's central region to improve the sensitivity. Establishing the use of internal electrodes on a stretchable textile needs further investigation into piezoresistive structure fabrication, measurement strategy, and calibration. This article presents a comprehensive study of an ERT-based tactile sensor with distributed electrodes. We describe key fabrication details of a layered textile-based piezoresistive structure, an iterative method for choosing the current injection pathways that yields pairwise optimal patterns, and a calibration process to account for the spatially varying sensitivity of such sensors. We demonstrate two sample sensors with electrodes located only on the boundary or distributed across the surface, and we evaluate their performance via three methods widely used to test tactile sensing in biological systems: single-point localization, two-point discrimination, and contact force estimation.

DOI Project Page [BibTex]

DOI Project Page [BibTex]


no image
COMPACT: Concurrent or Ordered Matrix-Based Packing Arrangement Computation Technique

Serhat, G.

Applied Sciences, 11(11):5217, June 2021 (article)

Abstract
Despite their versatility in treating irregular geometries, the raster methods have received limited attention in solving packing problems involving rotatable objects. In addition, raster approximation allows the use of unique performance metrics and indirect consideration of constraints, which have not been exploited in the literature. This study presents the Concurrent or Ordered Matrix-based Packing Arrangement Computation Technique (COMPACT). The method allows the objects to be rotated by arbitrary angles, unlike the right-angled rotation restrictions imposed in many existing packing optimization studies based on raster methods. The raster approximations are obtained through loop-free operations that improve efficiency. Additionally, a novel performance metric is introduced, which favors efficient filling of the available space by maximizing the overall contact within the domain. Moreover, the objective functions are exploited to discard the overlap and overflow constraints and enable the use of unconstrained optimization methods. The results of the case studies demonstrate the effectiveness of the proposed technique.

DOI [BibTex]


no image
Free and Forced Vibration Modes of the Human Fingertip

Serhat, G., Kuchenbecker, K. J.

Applied Sciences, 11(12):5709, June 2021 (article)

Abstract
Computational analysis of free and forced vibration responses provides crucial information on the dynamic characteristics of deformable bodies. Although such numerical techniques are prevalently used in many disciplines, they have been underutilized in the quest to understand the form and function of human fingers. We addressed this opportunity by building DigiTip, a detailed three-dimensional finite element model of a representative human fingertip that is based on prior anatomical and biomechanical studies. Using the developed model, we first performed modal analyses to determine the free vibration modes with associated frequencies up to about 250 Hz, the frequency at which humans are most sensitive to vibratory stimuli on the fingertip. The modal analysis results reveal that this typical human fingertip exhibits seven characteristic vibration patterns in the considered frequency range. Subsequently, we applied distributed harmonic forces at the fingerprint centroid in three principal directions to predict forced vibration responses through frequency-response analyses; these simulations demonstrate that certain vibration modes are excited significantly more efficiently than the others under the investigated conditions. The results illuminate the dynamic behavior of the human fingertip in haptic interactions involving oscillating stimuli, such as textures and vibratory alerts, and they show how the modal information can predict the forced vibration responses of the soft tissue.

DOI Project Page [BibTex]


no image
Design of Circular Composite Cylinders for Optimal Natural Frequencies

Serhat, G.

Materials, 14(12):3203, June 2021 (article)

Abstract
This study concerns optimizing the eigenfrequencies of circular cylindrical laminates. The stiffness properties are described by lamination parameters to avoid potential solution dependency on the initial assumptions of the laminate configurations. In the lamination parameter plane, novel response contours are obtained for the first and second natural frequencies as well as their difference. The influence of cylinder length, radius, thickness, and boundary conditions on the responses is investigated. The lamination parameters yielding the maximum response values are determined, and the first two mode shapes are shown for the optimum points. The results demonstrate that the maximum fundamental frequency points of the laminated cylinders mostly lie at the inner lamination parameter domain, unlike the singly curved composite panels. In addition, the second eigenfrequency shows a nonconvex response surface containing multiple local maxima for several cases. Moreover, the frequency difference contours appear as highly irregular, which is unconventional for free vibration responses.

DOI [BibTex]


Driving Skill Modeling Using Neural Networks for Performance-Based Haptic Assistance
Driving Skill Modeling Using Neural Networks for Performance-Based Haptic Assistance

Lee, H., Kim, H., Choi, S.

IEEE Transactions on Human-Machine Systems, 51(3):198-210, IEEE, June 2021 (article)

Abstract
This article addresses a data-driven framework, modeling expert driving skills for performance-based haptic assistance using neural networks (NNs). We have built a haptic driving training simulator to collect expert driving data and to provide proper haptic feedback. We establish an expert driving skill model by training NNs with the collected data. Then, the skill model is applied to the performance-based haptic assistance to provide optimized references of the steering/pedaling movements. We evaluate the skill model and its application to the performance-based haptic assistance in two user experiments. The results of the first experiment demonstrate that our skill model has appropriately captured experts’ steering/pedaling skills. The results of the second experiment show that our performance-based haptic assistance can help novice drivers perform steering as expert drivers, but cannot assist their pedaling performance.

DOI [BibTex]

DOI [BibTex]