Project Intro:
Graduation Project at BEng Industrial Engineering, University of Electronic Science and Technology of China
KEYWORDS: Exoskeleton Robot; Tactile Stimuli; Information Feedback, Human-computer Bidirectional Interaction.
METHOD & TOOLS: Experiment Design; Quantitative Research ; User Interview; Python; C; Arduino; STM32.
DURATION: 6 months.
INSTITUTION:
Abstract
For users of exoskeleton robots, it is very important to be able to obtain the state of motion of the lower extremities accurately in real time. Information feedback based on tactile stimuli can well meet this requirement. For the purpose of finding an optimal feedback method, this paper designs two sets of vibration feedback devices and three basic experiments to explore different vibration modes and stimulation sites. The paraplegic patient can obtain a tactile re-mapping at the perceptual site of the upper limb to simulate the sense of touch and proprioception of the lower extremity during walking, and ultimately achieve the effect of human-computer bidirectional interaction.
Background
In 2010, the total number of disabled people in China was 85.02 million, among which 24.72 million were physically disabled. It is estimated that more than 8 million patients with paraplegia and hemiplegia. For these patients, the traditional treatment and maintenance method is by professional nurses to help them recover and train daily to prevent diseases such as muscle atrophy. However, with the increase in the number of lower extremity disabilities and the limited energy of rehabilitation trainers, they can no longer meet the demand, exoskeleton robots emerged in this specific situation.
Although these patients have both somatosensory and motor deficits, few studies have restored both motor and somatosensory feedback in paraplegia patients. In the absence of haptic feedback, patients have to rely mainly on vision to control exoskeleton robot . In these cases, the user does not receive any tactile or proprioceptive feedback related to the state of the device. Especially when the patient is walking with the exoskeleton robot, they need to look at the floor every time when the device touches the ground, which causes a lot of inconvenience.
The key problem to be solved is to find the optimal feedback mode that can accurately convey the lower extremity motion state of the exoskeleton in real time. The goal is to build an exoskeleton motion state feedback system based on tactile stimulation. The research method of human factor engineering and human-computer interaction is mainly used. In the early stage, it is intended to explore the feedback accuracy of different positions, different intensity and different stimulus modes, and finally obtain an optimal, real-time, accurate and stable interaction mode, so as to improve the safety and user experience of exoskeleton robot.
Experiment Design
＃Overview
The whole experimental platform consists of hardware platform and software platform. The hardware platform is responsible for providing feedback, while the software platform is responsible for providing hardware control and data acquisition in the experiment.
＃Tactile Interaction Design
Due to the complexity of human gait, in the design of experiment, this paper selects the step length and step height as two spatial gait parameters feedback inputs. The feedback output are the tactile stimulus generated by vibration motors placed on human body parts. Based on the consistency under the same reference frame, the back of the frontal plane consistent with the change of height was selected as the feedback part of step height, and the lateral arm of the sagittal plane consistent with the change of horizontal displacement was selected as the feedback part of step length.
＃Devices Design
Since the change of step height and step length is a continuous change, it is necessary to find a feedback mode that can feedback the vibration information of continuous change. For detailed tactile information, hairless skin, the palm is the most effective place to feel. The hairy skin can effectively detect the magnitude of vibration and force, and can be used to present spatial tactile cues at various skin locations. The variation of vibration intensity and the spatial information feedback of different vibration stimuli meet the requirements. Therefore, the following two tactile feedback devices are designed in this paper.
The Linear Feedback System
It consists of 14 vibrators distributed on the back and arm (4 on the left and 4 on the right backs, 3 on the left and 3 on the right arms). The back and arm vibrators form four feedback arrays in a straight line: the step height and the step length of the left and right lower limbs respectively.
Feedback of step height: when users start walking movement, lift the foot, step higher began to rise, located in the back of leg information feedback is started from the feedback feedback array array the vibrator at the bottom of the vibration, then increase with the last step is high, in the higher position, beginning with a vibrator vibration when step high maximum, located in the highest position in the array vibrator to vibration. As the step height decreases, the active vibrator in the array also decreases in spatial position until the moving foot lands and stops vibrating. When the other moving foot is raised, the feedback array on the corresponding side vibrates in the same way.
Feedback of step length: when users begin walking movements, lift the foot, step began to increase, in the arms of feedback is movement of the leg information feedback array from the feedback array end vibrator began vibration, along with the step continues to increase, in the higher position of vibrator vibration, high step when achieving the maximum, is located in the array far end position of the vibrator to vibration. Until the moving foot hits the ground and stops vibrating. When the other moving foot is raised, the feedback array on the corresponding side vibrates in the same way.
Feedback of step height: when users start walking movement, lift the foot, step higher began to rise, located in the back of leg information feedback is started from the feedback feedback array array the vibrator at the bottom of the vibration, then increase with the last step is high, in the higher position, beginning with a vibrator vibration when step high maximum, located in the highest position in the array vibrator to vibration. As the step height decreases, the active vibrator in the array also decreases in spatial position until the moving foot lands and stops vibrating. When the other moving foot is raised, the feedback array on the corresponding side vibrates in the same way.
Feedback of step length: when users begin walking movements, lift the foot, step began to increase, in the arms of feedback is movement of the leg information feedback array from the feedback array end vibrator began vibration, along with the step continues to increase, in the higher position of vibrator vibration, high step when achieving the maximum, is located in the array far end position of the vibrator to vibration. Until the moving foot hits the ground and stops vibrating. When the other moving foot is raised, the feedback array on the corresponding side vibrates in the same way.
The Single Point Feedback System
The tactile feedback device consists of four vibrators distributed in the back and arm (one on the left back and one on the right back, one on the left arm and one on the right arm, in total). The back and arm vibrators form four feedback points: step height and step length of left and right lower limb movement gait.
Feedback of step height: when users begin walking movements, lift the foot, step higher began to rise, is located in the back of the feedback movement of the leg starts at information feedback to the strength of the tiny vibration, increase with the last step is high, the vibration of the vibrator strength is also growing, high step when achieving the maximum, the strength of the vibrator is maximum. As the step height decreases, the vibrator's strength decreases until the moving foot hits the ground and the vibration stops. The other moving foot is lifted and the feedback point on the corresponding side vibrates in the same way.
Feedback of step length: when users begin walking movements, lift the foot, step began to increase, in the arms of feedback is movement of the leg starts at information feedback to the strength of the tiny vibration, along with the step continues to increase, the vibration of the vibrator strength is also growing, high step when achieving the maximum, the strength of the vibrator is maximum. Until the moving foot hits the ground and stops vibrating. When the other moving foot is raised, the feedback array on the corresponding side vibrates in the same way.
Feedback of step height: when users begin walking movements, lift the foot, step higher began to rise, is located in the back of the feedback movement of the leg starts at information feedback to the strength of the tiny vibration, increase with the last step is high, the vibration of the vibrator strength is also growing, high step when achieving the maximum, the strength of the vibrator is maximum. As the step height decreases, the vibrator's strength decreases until the moving foot hits the ground and the vibration stops. The other moving foot is lifted and the feedback point on the corresponding side vibrates in the same way.
Feedback of step length: when users begin walking movements, lift the foot, step began to increase, in the arms of feedback is movement of the leg starts at information feedback to the strength of the tiny vibration, along with the step continues to increase, the vibration of the vibrator strength is also growing, high step when achieving the maximum, the strength of the vibrator is maximum. Until the moving foot hits the ground and stops vibrating. When the other moving foot is raised, the feedback array on the corresponding side vibrates in the same way.
The height and distance of the lower limbs were distinguished by colour, and the left and right feet were distinguished by texture. In the intensity change feedback device, the darker the colour is, the greater the vibration intensity is, while the lighter the colour is, the weaker the vibration intensity is. In the displacement feedback device, the X-axis is the sequential state stage of the gait, and the Y-axis is the height and horizontal distance of the gait.
＃Prototype
Single-point feedback system consists of 4 vibration motors, 1 driver chip(ULN2003A) and 1 micro-controller(Arduino UNO) ;
Linear feedback system consists of 14 vibration motors, 2 driver ship(ULN2003A) and 1 micro-controller(STM32F103Z).
＃Experiment
The goal of this paper is to find an optimal way of lower limb movement information feedback for two kinds of vibration feedback above design, need feedback respectively on the basis of human cognitive tests, and according to the basic feedback cognitive experiment result will feedback on ways to improve after using two feedback methods for simulation of the real gait usability testing.
A total of 10 subjects (5 male and 5 female) were recruited in the experiments, all undergraduates and postgraduates of UESTC. The 10 subjects were all normal people with known physical and mental diseases. The average age was 22 years old, the average height was 170cm, and the average weight was 55kg.
In this experiment, the two parameters used to evaluate the cognitive ability of the subjects were the accuracy of the feedback response and the Reaction Time. Reaction time (RT) is one of the earliest and most widely used reaction variables in psychological experiments. The response does not occur at the same time as the stimulus is given. Reaction time refers to the time interval between the presentation of the stimulus and the beginning of the reaction.
EXP 1 - Before the formal cognitive experiment, it is necessary to measure the basic reaction time of each subject. The basic reaction time can be used as a Base Line of the data of subsequent experiments, so as to eliminate the data with too large gap with the Base Line and achieve the purpose of screening the effective experimental data.
EXP 2 -In the experiment, the subjects will feel the change of vibration intensity from a single vibrator on the arm and give feedback. the stimulation is divided into 70 groups of randomly arranged vibration intensity changes (enhanced or weakened). One group of vibration intensity changes is composed of two vibration intensities ranging from 1 to 9, with the change interval of 1s. Subjects need to click the "enhance" or "weaken" button in the fastest time after they feel the change of intensity. After clicking the button, the interface will return the value of the option and the response time for making the choice
The single test lasted about 20 minutes, and participants completed three tests over five days (once every other day).
The single test lasted about 20 minutes, and participants completed three tests over five days (once every other day).
EXP 3 - Participants will experience the feedback of motor array composed of three vibrators and give response, stimulation to 1 set of random vibration intensity change (forward or backward), a set of vibration intensity change consists of three vibrator vibration of different order, change interval is 0.3 s. The subjects need to click the "Forward" or "Back" button in the fastest time after they feel the change of mode. After clicking the button, the value of the option and response time will be returned to the interface. The single test lasted about 10 minutes, and participants completed three tests (every other day) over a period of five days.
EXP 4 - According to the gait curve, the change of height and horizontal displacement is converted into the analog signal that can be recognized by the vibration device, and the lower limb motion state of the user of the exoskeleton robot is reflected by vibration. The simulated gait was divided into four stages -- right foot swinging, right foot falling to the ground and left foot lifting, left foot swinging and left foot falling to the ground and right foot lifting. At the beginning of the experiment, a vibration mode of the gait phase was randomly given. At this time, the subjects had no visual feedback. At the end of the vibrator, the subjects chose a gait phase that they thought was correct.
In order to simulate the change of real gait as much as possible, it is necessary to collect the data of real gait. In this paper, 20 normal walking posture data from Vicon motion capture system are collected, among which the gait with better data is selected for simulation.
Since the ankle joint is consistent with the spatial position of the foot during walking, the height and horizontal displacement of the ankle joint are selected as the input of feedback information in this paper.
The vibration device can only feedback the change of state in a discrete way, so the input state quantity information should be processed, and the information should be converted into a form that can be recognized by the hardware.
Before the formal testing, use the program to simulate walking gait and derive from the Vicon system model synchronization, the subjects watching the moving images for visual feedback at the same time feel the vibration feedback, will walk the vibration of the vibrator is mapped to the actual action, through the establishment of a mental model for the next test as a test basis.
Results