EMG-based simultaneous and proportional estimation of wrist/hand kinematics in uni-lateral trans-radial amputees
© Jiang et al.; licensee BioMed Central Ltd. 2012
Received: 21 October 2011
Accepted: 15 May 2012
Published: 28 June 2012
We propose a method for estimating wrist kinematics during dynamic wrist contractions from multi-channel surface electromyography (EMG). The algorithm extracts features from the surface EMG and uses dedicated multi-layer perceptron networks to estimate individual joint angles of the 3 degrees of freedom (DoFs) of the wrist. The method was designed with the aim of proportional and simultaneous control of multiple DoFs of active prostheses by unilateral amputees. Therefore, the proposed approach was tested in both unilateral transradial amputees and in intact-limbed control subjects. It was shown that the joint angles at the 3 DoFs of amputees can be estimated from surface EMG recordings , during mirrored bi-lateral contractions that simultaneously and proportionally articulated the 3 DoFs. The estimation accuracies of amputee subjects with long stumps were 62.5% ± 8.50% across all 3 DoFs, while accuracies of the intact-limbed control subjects were 72.0% ± 8.29%. The estimation results from intact-limbed subjects were consistent with earlier studies. The results from the current study demonstrated the feasibility of the proposed myoelectric control approach to provide a more intuitive myoelectric control strategy for unilateral transradial amputees.
The Electromyographic signal (EMG) has long been used to control powered prostheses, particularly those for upper-extremities, because it contains information on the neural control of movement. This control scheme is often referred to as myoelectric control. In the past 30 years, it has been demonstrated that pattern classification of EMG signals can consistently achieve very high classification accuracy. For example, an accuracy greater than 95% can be achieved when classifying 6 hand/wrist contractions of intact-limbed subjects from EMG signals[4, 5]. The performance of these approaches on amputees is also promising. Ajiboye and Weir reported classification accuracy ranging from 74% to 99%, for two amputee subjects performing 5 wrist functions. Employing a mirrored training paradigm and artificial neural network (ANN), Sebelius et al. reported that both traumatic and congenital amputees can achieve > 80% accuracy on selected 4-6 wrist/hand movements. More recently, Li et al. reported > 88% accuracy in 5 transradial amputees over 6 wrist/hand motions.
Despite the promising performance of these myoelectric control algorithms, currently no commercial myoelectric prosthesis is based on EMG pattern classification. Rather, commercial devices are still based on the conventional, very simple direct control approach (based solely on EMG amplitude), which has been used for more than half century. There are many reasons for this contrast between the industrial and academia state-of-the-art.
Although extensive research activities aimed at improving applicability of pattern classification based myoelectric control, such as feature stability, training adaptation[10, 11], and false positive (activation) control, the pattern classification paradigm is drastically different from the way in which the neuromuscular system controls the muscles. A pattern classification paradigm indeed provides a sequential and on/off control of a predefined set of muscle activation patterns (motions). In contrast, the neuromuscular system smoothly articulates (proportional control) multiple degree-of-freedoms (DoFs) simultaneously (simultaneous control). As such, the prosthesis users often feel that the provided control is not intuitive, resulting in long, some times discouraging training/learning periods.
To realize a bio-mimic and more intuitive myoelectric control paradigm, the control algorithms should provide simultaneous and proportional control of multiple DoFs, as it has been recently addressed in a few studies. Based on the concept of muscle synergy, Jiang et al. demonstrated that a modified non-negative matrix factorization algorithm can simultaneously estimate the torque produced at the three DoFs of the wrist in intact-limbed subjects. Nielsen et al. extended the method proposed by Jiang et al.. In that study, a bilateral, mirror-training strategy was employed, and it was shown the force from contra-lateral limb can be estimated reliably using the EMG from the ispi-lateral limb, during mirror movements. This is fundamental for the application in unilateral amputees where the training of the algorithm obviously can not be performed between EMG and force recorded from the phantom limb. Further, Muceli et al.[16, 17] demonstrated in intact-limbed subjects that mirror movements can be used for training simultaneous and proportional control in dynamic tasks to estimate joint kinematics instead of force.
The current study aims at developing a method for proportional and simultaneous control of three DoFs of the wrist joint (flexion/extension, radial/ulnar deviation, and supination/extension), and at demonstrating its clinical applicability in unilateral amputees. For this purpose, both transradial amputee subjects (with different levels of amputations), and intact-limbed control subjects were tested using an experimental protocol where multi-channel surface EMG was used to estimate the joint kinematics (joint angles) during mirrored bilateral, simultaneous articulations of the wrist.
Description of participating subjects
Amputee subjects data
TIME OF AMP.
POSITION OF AMP.
5cm distal from elbow
ca .10cm distal from elbow
2/3 distal from elbow
ca. 20cm distal from elbow
ca. 20cm distal from elbow
In addition to the 6 amputee subjects, 5 able-bodied subjects (2 male, 3 female; age range: 24-40 years; all right-handed, H1 - H5), with no known neuromuscular disorders, also took part in the experiments, as control subjects. This subject group was denoted control group (CG). All 11 subjects signed an informed consent form prior to participating in the experiments. The experimental protocol was approved by the local ethics committee.
The study involved the concurrent recording of multichannel surface EMG signals of upper limbs and kinematics of unrestrained and dynamic, non-loaded contractions of hand/wrist during mirrored bilateral wrist movements.
For all subjects, 7 pairs of Ag-AgCl surface bipolar electrodes (Type: Ambu NeuroLine 720) were placed on each forearm. At the intact side, the electrode pairs were placed around the thickest part of the forearm (approximately 1/3 distally from the elbow), equi-spaced in a circle around the forearm, similarly to[11, 14]. Equi-spaced electrode placement was used, rather than targeting at specific muscles, because 7 electrode pairs provided necessary coverage of the area of interest, as shown in previous studies[14, 15]. The first pair was placed 1 cm medially from the ulnar bone (found by palpation) and the remaining six pairs were positioned sequentially in the pronation direction. On the amputated side, the electrodes were placed on the same place as at the intact side, whenever possible. When the stump was too short, the electrodes were placed around the region where most musculature existed (found by palpation). A reference armband (placed on one of the wrists for the intact-limbed subjects and on the wrist of the intact side of the amputees) was used for common reference point. All electrodes were connected via high noise-rejection cables to an EMG-amplifier (EMG-USB, 128 channel, OT Bioelettronica), where the EMG signals were sampled at 2048 Hz, amplified at 2k, and digitized with 12-bit precision.
During an experimental session the subject sat in a standard chair, placed on a podium 0.3 m above floor level, with the elbows resting on two armrests, as shown in Figure1. The armrests were adjusted so that the subject felt relaxed, and his/her shoulders and upper arms were in symmetric positions. The eight cameras of the motion capture system were mounted on tripods, at various heights (0.5 - 2.5 m) above the floor, and placed in a circular pattern (average radius 2.5 m) around the subject. The exact positions and heights of the cameras were optimized during preliminary tests to provide the maximum coverage of all markers during the intended movements.
List of the contractions performed
1: flexion/extension (DoF1)
along a single DoF
2: radial/ulnar deviation (DoF2)
(freq. ≈ 0.5 − 1 Hz)
3: pronation/supination (DoF3)
4: DoF1 + DoF2
of two DoFs, in which
5: DoF2 + DoF1
one DoF was articulated
6: DoF1 + DoF3
sinousoidally, and the other
7: DoF3 + DoF1
was fixed at positions close
8: DoF2 + DoF3
to maximal range of motion
9: DoF3 + DoF2
Cyclic contractions of
10: DoF1 + DoF2 + DoF3
(freq. ≈ 0.5 − 1 Hz)
The EMG signal was band pass filtered (10 - 450 Hz, second order Butterworth filter). To estimate the kinematics at the wrist joint, the time domain (TD) feature set, and the 6 autoregressive coefficients (AR), namely the TDAR feature set, were used. The analysis window was 100 ms long, with 60 ms overlap. In preliminary analyses (not reported) of this study, more complex features, such as wavelet marginals or coefficients, did not provide significantly better estimation performance. Therefore, only the TDAR feature set was used, both for the amputee subjects and for the able-bodied subjects.
Kinematic data processing
Multi-layer perceptron network training
Multi-layer perceptron (MLP) artificial neural networks were used to learn the association between the EMG features and the kinematic signals. In this study, a ’dedicated MLP approach’ was taken in estimating the simultaneous articulations of the 3 DoFs, similar to the approach described in. It was shown that this approach is superior than the ’one MLP for all DoFs’ approach in[14, 15].
To investigate the possibility of using EMG from one arm to estimate the arm kinematics of the other arm (as is necessary for unilateral amputees), two MLP training scenarios were performed: contra-lateral training and ipsi-lateral training, as it was done for intact-limbed subjects in[15, 16]. For contra-lateral training, the MLPs were trained using EMG features from the amputated side (for amputee subjects), or the dominant side (intact limbed subjects). For ipsi-lateral training, the MLPs were trained using EMG features obtained from the intact side (for amputee subjects), or from the non-dominant side (for intact limbed subjects).
In addition, to investigate the ability of the MLP to estimate the kinematics at different DoFs, the MLPs were trained and tested on various data sets where the DoFs were activated selectively and/or simultaneously. This determined 4 analysis scenarios based on the DoF(s) considered. For example, the analysis scenario was identified as DoF12 when only contractions involving DoF1 and DoF2 were used for analysis (contractions 1, 2, 4, and 5; estimating for the first two DoFs). Similarly, the analysis scenario DoF13 consisted of the analysis of the contractions (numbered as 1, 3, 6, and 7), where only DoF1 and DoF3 were activated. When all 10 contraction types were used in the analysis, the scenario is indicated as DoF123, and corresponded to the estimation of all DoFs simultaneously.
The reason for using R2 as the performance metric over mean square error (MSE), is that MSE produces a biased result when the target values are very small. In the current study, both the DoF-wise performance in (5) and the global performance in (4) are reported.
The main hypothesis is that the performance in the contra-lateral training scenario in amputees is comparable to that of intact-limbed subjects. Separate ANOVA tests were conducted for each of the 4 DoF-wise scenarios, namely, DoF 12, DoF 13, DoF 23, and DoF 123. In these tests, the response variables were R2, and the two factors were Contra-/Ipsi- Lateral and Subject Group (SS/LS/CG). The subjects were nested with the Subject Group factor, and were repeated measures within Contra/Ipsi-lateral factor. The significance level was set at 95%. As shown in the Results Section below, for all the tests ran, there was always a strong and significant interaction between the two factors, in which case it is not advisable to interpret the main effects directly. Therefore, the effect of Contra-/Ipsi-lateral was analyzed by fixing the Subject Group factor, and the effect of Subject Group factor was analyzed by fixing the Contra-/Ipsi-lateral factor, with one-way ANOVA analyses.
The R2 value for DoF 123 of the three subject groups were: SS 19.7% ± 5.76% (77.5% ± 4.21%), LS 62.5% ± 8.50% (79.3% ± 10.4%) and CG 72.0% ± 8.29% (73.6% ± 8.59%), for Contra-lateral (Ipsi-lateral) scenarios, respectively. All results are summarized in Figure5. It is worth noting that for almost all training scenarios from a particular subject, the coefficient of variation of the R2 within any 5-fold cross validation was usually smaller than 10%. The only exceptions were for the SS Subject Group in Contra-Lateral scenario, where the performance were poor. This small variation within the cross-validation indicated a rather homogeneous data sets.
As discussed above, all the two-way ANOVAs revealed significant interactions between the two main factors. Therefore, separate one-way ANOVAs were performed for each factor, by fixing the level of the other factor, with an increased significance level at 97.5% (Bonferroni correction).
Subject group effect
When the Contra-/Ipsi- Lateral factor was fixed at ipsi-lateral, the performance across the groups was not statistically significant for DoF123 (p = 0.10), DoF12 (p = 0.15), and DoF13 (p = 0.12). But there was significant difference for DoF23 (p = 0.002). Post-hoc analysis for this case showed that the LS group performed significantly better than the CG and SS groups, while no significant difference existed between the CG and SS groups. These results indicated that, in most cases, there was no difference across the subject groups when ipsi-lateral EMG was used to estimate the joint angles, as it was expected (the control condition).
Contra-/Ipsi lateral effect
This paper is the first that demonstrates the feasibility of estimating wrist/hand kinematics of transradial amputees using surface EMG during mirrored bi-lateral movements with simultaneous and proportional activations of the 3 wrist DoFs. Previous research by our group[15–17] showed that this is feasible in case of intact-limbed subjects. When a simplified version of the proposed training strategy (limited to two DoFs and to isometric contractions, with one MLP for all DoFs) was applied to one subject with congenital malformation of the forearm, performance worsened, but was still encouraging. The present paper provides a systematic validation of mirror training for estimating wrist kinematics in amputee subjects. The proposed strategy was tested in individuals with different levels of amputation.
The results on intact-limbed subjects in the current study are consistent with previous studies. For the CG group and DoF12, the R2 values were 88.5% ± 4.28% (90.6% ± 3.68%) in contra-lateral (ipsi-lateral) cases. The corresponding values in were 90% ± 2% and 93% ± 2%. The performance worsened in DoF123 scenario, with R2 of 72.2% ± 8.29% (73.6% ± 8.59%) in contra-lateral (ipsi-lateral) cases. This result, which is also in agreement with previous work, is likely explained by the fact that the pronators and supinators are deep muscles whose activities can be masked by flexors and extensors which are superficial. No differences were found between contra-lateral and ipsi-lateral training scenarios for the intact-limbed subjects.
For amputees subjects, we investigated performance both for the intact and amputated sides, in order to allow a direct comparison with able bodied subjects. As expected, amputees performed better with their intact limbs. As shown in Figure6, no significant differences were found for the performance among intact-limbed subjects and the intact side of amputees, with the only exception for the scenario DoF23 where LS amputees performed better than the other two groups.
For amputees with long stumps (LS group), the contra-lateral estimation accuracy (R2) was 62.5% ± 8.50% for DoF123. This was lower than the CG group (72.2% ± 8.29%), which is explained by noting that the amputee subjects have not been using the remnant muscles on their amputated side for such complex motor tasks after the amputation. Among all the scenarios, the LS group showed the worst performance for tasks involving radial/ulnar deviation. The values of this DoF were: 60.6% ± 2.34% , 57.0% ± 9.52%, 41.3% ± 3.32% for DoF12 DoF23, and DoF123, respectively. However, for the intact-limbed subjects, pronation/supination was consistently the worst estimated DoF, as also noted in previous studies[14, 16]. Figure5 confirmed that pronation/supination was better estimated than radial/ulnar deviation in the LS group. Whether this counter-intuitive result is inherent to limb deficiency or only due to small sample size needs further investigation.
The striking differences between the performances of the SS and the LS group in contra-lateral scenario highlights the myoelectric control paradox: higher amputation requires more functional restoration, while at the same time leaves less signal sites to acquire EMG signals. Subjects A1 and A2 had stumps approximately 5 and 10 cm long, i.e. they did not retain enough remnant muscles in the forearm from which neural control information could be extracted. While the subject with the congenital malformation (A3) has a longer stump, she reported difficulties in imagining the movements with the malformed limb since this was the first time the subject was asked to do such tasks.
Taken together, our results showed that as long as sufficient muscles remain in the stump of transradial amputees, surface EMG recorded from the stump can be used to extract simultaneous and proportional control information for all 3 DoFs of the wrist. In particular, the LS group performed similarly to the CG group when only DoF1 and DoF3 were considered, which are functionally more important than DoF2. It is possible they could achieve even better performance after repetitively using the proposed training paradigm. For individuals with shorter stumps, it is needed to increase the amount of information for control, e.g. by accessing the peripheral nervous system with intra-neural electrodes, or by targeted muscle reinnervation.
Most of the previous myolectric control strategies that have been tested in amputees rely on the pattern classification paradigm (to estimate a set of classes), so a quantitative comparison of performance is not feasible. Reported classification accuracies are usually high (> 90%). Although the performance reported in this study seems lower (but not comparable anyway given the different metrics), it is worth noting that we evaluated performance during dynamic contractions while classification accuracy is usually calculated on static contractions. Moreover, our control strategy provides simultaneous and proportional control of multiple DoFs and not sequential discrimination of a fixed number of tasks. In addition to higher functionality, this control approach is also more intuitive than those based on pattern classification in that it provides continuous and simultaneous control of functions. Currently, the performance was assessed through offline processing of pre-recorded signals. However, the proposed system is suitable for real-time application: the analysis window was limited to 100 ms, the features (TDAR) were simple, and only 7 channels were used.
To further validate the clinical applicability of the proposed approach, the next step is to establish a relationship between the algorithmic performance (e.g. R2) and functional performance, or usability, from which an acceptable R2 value for prosthetic control can be established. In pattern recognition based myoelectric control, it was shown that the algorithmic performance (classification accuracy), at the very best, has a very weak correlation with the usability. This means that a system with 95% classification accuracy does not necessarily have a better usability than a system with 90% accuracy. Since the current approach allows intuitive, simultaneous and proportional control directly on the physiological DoFs, it is possible that a stronger relationship between the algorithmic performance and usability can be found. This issue will be the subject of further investigation.
For clinical feasible applications of the proposed approach, the DoF of hand open/close has to be included. This is because this DoF is the most important function for amputees. In a recent study, we have demonstrated that, for intact-limbed subjects, the kinematics of this DoF can be estimated with similar accuracies as the other DoFs.
Other ongoing work is underway to provide an online implementation of the proposed system to evaluate its real time performance on both amputees and intact-limbed subjects, and its clinical applicability.
This work is financially supported by the following sources: German Ministry for Education and Research (BMBF) via the Bernstein Focus Neurotechnology (BFNT) Göttingen under Grant No. 01GQ0810; European Commission via the Industrial Academia Partnerships and Pathways (IAPP) under Grant No. 251555 (AMYO); a Ph.D. fellowship from Regione Autonoma della Sardegna Italy.
- Farina D, Merletti R, Enoka RM: The extraction of neural strategies from the surface EMG. J Appl Physiol 2004,96(4):1486-1495. 10.1152/japplphysiol.01070.2003View ArticlePubMedGoogle Scholar
- Parker P, Scott R: Myoelectric prostheses: state of the art. J Med Eng Technol 1988,12(4):143-151. 10.3109/03091908809030173View ArticlePubMedGoogle Scholar
- Jiang N, Falla D, d’Avella A, Graimann B, Farina D: Myoelectric control in neurorehabilitation. Crit Rev Biomed Eng 2010,38(4):381-391. 10.1615/CritRevBiomedEng.v38.i4.30View ArticlePubMedGoogle Scholar
- Huang Y, Englehart K, Hudgins B, Chan A: A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Trans Biomed Eng 2005,52(11):1801-1811. 10.1109/TBME.2005.856295View ArticlePubMedGoogle Scholar
- Hargrove L, Englehart K, Hudgins B: A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans Biomed Eng 2007,54(5):847-853.View ArticlePubMedGoogle Scholar
- Ajiboye AB, Weir RF: A heuristic fuzzy logic approach to EMG pattern recognition for multifunctional prosthesis control. IEEE Trans Neural Syst Rehabilitation Eng 2005,13(3):280-291. 10.1109/TNSRE.2005.847357View ArticleGoogle Scholar
- Sebelius FC, Rosén BN, Lundborg GN: Refined myoelectric control in below-elbow amputees using artificial neural networks and a data glove. J Hand Surg 2005,30(4):780-789. 10.1016/j.jhsa.2005.01.002View ArticleGoogle Scholar
- Li G, Schultz A, Kuiken T: Quantifying pattern recognition based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehabilitation Eng 2010,18(2):185-192.View ArticleGoogle Scholar
- Tkach D, Huang H, Kuiken T: Study of stability of time-domain features for electromyographic pattern recognition. J NeuroEngineering Rehabilitation 2010, 7: 21. 10.1186/1743-0003-7-21View ArticleGoogle Scholar
- Sensinger J, Lock B, Kuiken T: Adaptive pattern recognition of myoelectric signals: exploration of conceptual framework and practical algorithms. IEEE Trans Neural Syst Rehabilitation Eng 2009,17(3):270-278.View ArticleGoogle Scholar
- Lorrain T, Jiang N, Farina D: Influence of the training set on the accuracy of surface EMG classification in dynamic contractions for the control of multifunction prostheses. J NeuroEngineering Rehabilitation 2011, 8: 25. 10.1186/1743-0003-8-25View ArticleGoogle Scholar
- Scheme E, Englehart K, Hudgins B: Selective classification for improved robustness of myoelectric control under nonideal conditions. IEEE Trans Biomed Eng 2011,58(6):1698-1705.View ArticlePubMedGoogle Scholar
- Tresch MC, Saltiel P, Bizzi E: The construction of movement by the spinal cord. Nat Neurosci 1999,2(2):162-167. 10.1038/5721View ArticlePubMedGoogle Scholar
- Jiang N, Englehart KB, Parker PA: Extracting simultaneous and proportional neural control information for multiple degree of freedom prostheses from the surface electromyographic signal. IEEE Trans Biomed Eng 2009,56(4):1070-1080.View ArticlePubMedGoogle Scholar
- Nielsen J, Holmgaard S, Jiang N, Englehart K, Farina D, Parker P: Simultaneous and proportional force estimation for multifunction myoelectric prostheses using mirrored bilateral training. Biomed Eng, IEEE Trans 2011,58(3):681-688.View ArticleGoogle Scholar
- Muceli S, Jiang N: Multichannel surface EMG based estimation of bilateral hand kinematics during movements at multiple degrees of freedom. Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE 2010, 6066-6069.View ArticleGoogle Scholar
- Muceli S, Farina D: Simultaneous and proportional estimation of HandKinematics From EMG during mirrored movements at MultipleDegrees-of-Freedom Neural Syst Rehabilitation Eng, IEEE Trans. PP(99) 2011, 1.Google Scholar
- Hudgins B, Parker P Scott RN: A new strategy for multifunction myoelectric control. 1993.Google Scholar
- Winter D: Biomechanics and Motor Control of Human Movement. New Jersey: Wiley and Sons; 2009.View ArticleGoogle Scholar
- Farrell T, Weir R: The optimal controller delay for myoelectric prostheses. IEEE Trans Neural Syst Rehabilitation Eng 2007, 15: 111-118.View ArticleGoogle Scholar
- d’Avella A, Portone A, Fernandez L, Lacquaniti F: Control of fast-reaching movements by muscle synergy combinations. J Neurosci 2006,26(30):7791-7810. 10.1523/JNEUROSCI.0830-06.2006View ArticlePubMedGoogle Scholar
- Rossini P, Micera S, Benvenuto A, Carpaneto J, Cavallo G, Citi L, Cipriani C, Denaro L, Denaro V, DiPino G, Ferreri E, Guglielmelli F, Hoffmann K, Raspopovic S, Rigosa J, Rossini L, Tombini M, Dario P: Double nerve intraneural interface implant on a human amputee for robotic hand control. Clin Neurophysiology 2010,121(5):777-783. 10.1016/j.clinph.2010.01.001View ArticleGoogle Scholar
- Kuiken TA, Miller LA, Lipschutz RD, Lock BA, Stubblefield K, Marasco PD, Zhou P, Dumanian GA: Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: a case study. The Lancet 2007,369(9559):371-380. 10.1016/S0140-6736(07)60193-7View ArticleGoogle Scholar
- Lock B: Design and Interactive Assessment of Continuous Multifunction Myoelectric Control Systems. In Master Thesis. University of New Brunswick; 2005.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.