- Open Access
Proportional estimation of finger movements from high-density surface electromyography
© The Author(s). 2016
- Received: 6 November 2015
- Accepted: 12 July 2016
- Published: 4 August 2016
The importance to restore the hand function following an injury/disease of the nervous system led to the development of novel rehabilitation interventions. Surface electromyography can be used to create a user-driven control of a rehabilitation robot, in which the subject needs to engage actively, by using spared voluntary activation to trigger the assistance of the robot.
The study investigated methods for the selective estimation of individual finger movements from high-density surface electromyographic signals (HD-sEMG) with minimal interference between movements of other fingers. Regression was evaluated in online and offline control tests with nine healthy subjects (per test) using a linear discriminant analysis classifier (LDA), a common spatial patterns proportional estimator (CSP-PE), and a thresholding (THR) algorithm. In all tests, the subjects performed an isometric force tracking task guided by a moving visual marker indicating the contraction type (flexion/extension), desired activation level and the finger that should be moved. The outcome measures were mean square error (nMSE) between the reference and generated trajectories normalized to the peak-to-peak value of the reference, the classification accuracy (CA), the mean amplitude of the false activations (MAFA) and, in the offline tests only, the Pearson correlation coefficient (PCORR).
The offline tests demonstrated that, for the reduced number of electrodes (≤24), the CSP-PE outperformed the LDA with higher precision of proportional estimation and less crosstalk between the movement classes (e.g., 8 electrodes, median MAFA ~ 0.6 vs. 1.1 %, median nMSE ~ 4.3 vs. 5.5 %). The LDA and the CSP-PE performed similarly in the online tests (median nMSE < 3.6 %, median MAFA < 0.7 %), but the CSP-PE provided a more stable performance across the tested conditions (less improvement between different sessions). Furthermore, THR, exploiting topographical information about the single finger activity from HD-sEMG, provided in many cases a regression accuracy similar to that of the pattern recognition techniques, but the performance was not consistent across subjects and fingers.
The CSP-PE is a method of choice for selective individual finger control with the limited number of electrodes (<24), whereas for the higher resolution of the recording, either method (CPS-PA or LDA) can be used with a similar performance. Despite the abundance of detection points, the simple THR showed to be significantly worse compared to both pattern recognition/regression methods. Nevertheless, THR is a simple method to apply (no training), and it could still give satisfactory performance in some subjects and/or simpler scenarios (e.g., control of selected fingers). These conclusions are important for guiding future developments towards the clinical application of the methods for individual finger control in rehabilitation robotics.
- Surface electromyography
- High-Density electrodes
- Machine learning
- Human–machine interfaces
- Rehabilitation robotics
- Finger control
- Hand rehabilitation
The dexterity of human hand is the result of complex motor patterns that generate a coordinated response of multiple muscles placed intrinsically in the hand and in the forearm. The control signals to move each finger of the hand are generated in separate regions of the primary motor cortex (M1) , and are delivered to the muscles via the efferent pathways of the spinal cord and peripheral nervous system . The neural commands elicit muscle electrical activity and a mechanical response. In recent years, it has been demonstrated that the intention to move the hand can be decoded using pattern recognition applied to recorded and processed electromyography (EMG) signals [3, 4]. This research was motivated by the importance to restore the hand function following amputation or an injury/disease of the nervous system, such as stroke. Most often, the aim was to detect less dexterous arm movements, such as the wrist rotations (e.g., pronation/supination) [5–7] and/or overall grasping patterns (e.g., palmar, lateral grip) [8, 9], whereas the classification and regression of finger movements has been less explored. Only recently, motivated by the development of modern dexterous hand prostheses  and hand exoskeletons [11–13], researchers started exploring the classification and regression of finger movements with the aim of establishing methods for intuitive control of these sophisticated systems, mimicking the dexterity of the human hand.
Journal papers on classification and regression of finger movements using electromyography
79 % (TR)
89 % (ND)
LDA, SVM, GMM
98 % (ND)
90 % (TR)
(JA) Norm RMS error 8–20 %
(JA) RMS error 0.085 rad −0.163 rad
Norm RMS error 7–14 %
(JA) Mean CORR
0.85 ± 0.07 (MCP)
0.78 ± 0.06 (PIP)
0.73 ± 0.04 (DIP)
14 – 16
(JA) R2 = 0.8
(FF) Norm RMS error 16 %
(FF) Norm RMS error 10–20 %
Recently, considerable attention has been devoted to investigating rehabilitation interventions which can facilitate the recovery of the sensory-motor functions impaired due to an injury/disease of the nervous system . Numerous studies [25–28] demonstrated that the motor ability could be regained through a task-specific intensive practice. In this context, robotic rehabilitation is a promising method for the restoration and relearning of motor functions, since it can provide mass practice in well-controlled conditions . Moreover, sEMG can be used to estimate the intention of the subject and operate the robot accordingly [12, 30–32]. This would create a user-driven control of a rehabilitation robot, in which the patient needs to provide a minimal activation to trigger and maintain the assistance. The benefit of this approach is that the subject is motivated to actively engage in therapy by recruiting his/her spared voluntary motor control, instead of passively relying on the robot to guide the movement . Furthermore, the EMG control allows highly disabled patients who cannot produce detectable forces and/or motions, but can generate residual EMG, to participate early in the user-responsive therapy. More specifically, the context for the present work is the rehabilitation of selective finger movements using a specialized hand rehabilitation robot (Amadeo, Tyromotion GmbH, AT). Rather than aiming at the simultaneous control of multiple fingers to achieve functional movements (e.g., grasps) as required in prosthetics, the emphasis here is on the selective activation of individual fingers (i.e., one finger at a time) while reducing the simultaneous false co-activations. The motivation for this approach is to promote relearning of the selective motor control skills, which are heavily impaired in neurological patients (e.g., stroke ).
The present study advances the state of the art of individual finger control by investigating proportional estimation of fingertip forces during tasks that combined different force profiles, force levels and rates of change of force. Three different methods based on common spatial filtering (CSP-PE), linear discriminant analysis (LDA) and simple thresholding (THR) were applied to learn the mapping from High-Density sEMG (HD-sEMG) to finger activation; their performance were compared in offline/online tests and across different numbers of electrodes. To the best of our knowledge, such a comprehensive set of conditions was not considered before in the context of single finger classification and regression (see Table 1).
The focus of the present study was on the estimation of the finger forces using HD-sEMG to record the electrical activity of the extrinsic hand muscles during isometric finger flexion and extension. The thumb, however, has a specific anatomy and a functional behavior, with an additional degree-of-freedom (opposition) fully controlled through the intrinsic muscles. Consequently, the present study considered only the four long fingers. Nevertheless, the thumb activation could be estimated as well by placing additional electrodes over the intrinsic muscles. This could be accomplished using conventional bipolar electrodes, and therefore, this was not relevant for the present study.
HD-sEMG was selected since it provides a high resolution of sensing points, capturing the high-fidelity spatial and temporal patterns of muscle activity and revealing a topographical map of focal activation areas corresponding to individual muscles. The muscle heads moving individual fingers are located close to each other, within the relatively small volume of the forearm . Therefore, HD-sEMG was chosen to selectively capture the individual muscle activity despite the significant spatial and temporal overlap. HD-sEMG has been used before to characterize the activity of the forearm muscles  . However, the present study represents the first application where an HD-sEMG interface has been applied for individual finger movement classification and regression, investigating a comprehensive set of conditions that were not considered before. The high resolution of the recording (192 channels) was exploited to assess the robustness of the tested methods with respect to the reduction in the number of electrodes, providing important perspectives regarding the potential practical applications. Moreover, to the best of our knowledge, there are no studies presenting an online protocol of finger control based on HD-sEMG, evaluating three control methods: one direct (THR), and two based on pattern recognition (LDA and CSP-PE). Furthermore, the two methods, CSP-PE and THR, have not been considered before for the control of individual fingers. All the experiments were conducted using a commercial rehabilitation robot, mimicking closely a real clinical context. In conclusion, this study presents some important insights for guiding future developments towards the clinical application of the methods for individual finger control in rehabilitation robotics.
The EMG signals were recorded using a High-Density 192-channel electrode grid (ELSCH064NM 3–3, OT Bioelettronica, 8x24 channels, 10 mm inter-electrode-distance, 8 × 24 cm) in a monopolar configuration placed on the dominant arm. The forearm length was measured in each subject using a measuring tape. The electrode array was positioned 6.4 ± 0.4 cm (25 % of the forearm length) from the elbow crease (Fig. 2c and d), covering 8 cm of the forearm longitudinally and 24 cm circumferentially. The electrode configuration allowed acquiring the sEMG activity of distal and proximal muscles such as the flexor digitorum superficialis and extrensor digitorium. The EMG signals were recorded using a multichannel electrophysiological amplifier (EMG-USB2, OT Bioelettronica, IT) connected to the host computer via a USB port. The gain was set to 500, the signals were band-pass filtered (eight order analog Bessel filter, bandwidth 10-750 Hz), sampled at 2048 Hz and digitally converted (12 bit A/D converter, 5 V dynamic range) with a resolution per least significant bit of 2.44 μV. The reference electrode was a ground strip placed at the distal end of the forearm, just next to the wrist joint.
The present study aimed at comparing three different proportional controllers (LDA, CSP-PE and THR) as they are applied to decode the activation of individual fingers from HD-sEMG patterns. The inputs were the processed EMG signals, and the regression methods outputted the estimated levels of finger activation proportional to the exerted force. The raw EMG data were segmented into a series of overlapping data analysis windows. The window length of 200 ms with 50 % overlap was selected since it represents a good trade-off between classification accuracy (CA) and controller delay . The Root Mean Square (RMS) was computed over the data window and used as an input for the classification/regression, since it is a time domain feature related to the force exerted by the muscle . Therefore, a class decision was produced for each data analysis window (every 100 ms), where the input for the regression was a vector of RMS values (one per electrode) computed over the 200-ms data window .
The LDA classification represents one of the most popular pattern recognition methods for myoelectric control. In summary, it models the distribution of the data within each class using a Gaussian distribution, where the means are estimated for each class individually and the covariance matrix is computed over the pooled data (shared covariance). The classification is performed by computing the class posterior probability (Bayes rule), which in the case of a shared covariance reduces to evaluating the linear discriminant functions separating the classes . As demonstrated by several studies, this simple and fast method performed similarly to or even better than the other, more complex non-linear pattern recognition methods for time-domain features [46, 47]. Since the LDA classifier has become the golden standard for the pattern recognition of EMG signals, it was selected as common benchmark. In this study once a class decision was taken, a proportional control value was extracted in a different way depending on the experiment. In the offline test, this value was extracted as an approximation of the recorded fingertip forces obtained by linear regression. In the online test, as the mean of the RMS values from a subset of channels related to the specific class, scaled to a percentage of MVC for the detected movement class.
Common Spatial Patterns (CSP) is a semi-supervised algorithm to determine a filter whose output has maximal and minimal variance when the multichannel input data come from the first and second class, respectively. Therefore, the filter maximizes the separation of the two classes based on the variance of the filter output signal. Commonly, CSP is used as a spatial filter for raw signals of two distinct classes, but there are several options for the extension to multi-class problems . The method has been originally used in brain-computer interfacing as a spatial filter for data preprocessing [49–51]. However, it has been recently adapted and tested for classification and regression in myoelectric control of prostheses, with promising results . In this study, we applied CSP as a proportional estimator (CSP-PE) in one vs. one configuration between all possible class pairs, as presented in . The CSP-PE has been selected since this is a novel method for myoelectric control with unique mathematical properties. Namely, the method aims at maximizing the contrast between classes and thereby minimizing the false co-activations. This indicates that it could be especially effective in the context of selective finger activation, which is the aim of the present work. In summary, the first step in the application of the CSP-PE was to determine the CSP filters for each pair of classes. The output of each filter was therefore tuned to maximize the response for the input data (feature vector) coming from one class, and respond minimally to the data from the other class. In the second step, the outputs of all the pairwise-class filters were fused in post-processing to estimate the class corresponding to the input data. Therefore, the activation of the class was determined by taking into account its relative contrast with respect to all the other classes. The selected class is the one that wins the competition, and the estimated activation reflects the uncertainty of this decision. For example, if the class loses at least one competition, the activation will be penalized leading to a small value. The raw outputs (activation levels) were scaled to yield a maximum value for 100 % of MVC. The method, including the original CSP formulation as well as the novel steps for CSP-based classification and regression, are given in detail in .
The THR is a simple method that involves direct control of each individual finger by identifying the focal areas of activity within the HD-sEMG interface. The THR was chosen because it is a simple method, easy to understand, implement (no training) and apply even by a non-technical personal, and thereby convenient for prospective practical application in clinical settings. The finger movement was detected if the activity at the selected subset of channels was above the predetermined threshold. THR was applied only during the online tests and a set of relevant channels was selected for each class (finger × movement) by the experimenter based on the visual observation of the EMG activity generated during the respective movements. The calibration trials were used to adjust the thresholds for each set of channels maximizing the correct classification. A class was recognized when the mean of the RMS values over the associated channel set crossed the threshold, and the RMS was higher than for the other classes. The proportional control value was the mean of the RMS values for the set of channels, scaled to a percentage of the MVC of each movement.
Nine healthy subjects (age between 26 and 41 years) participated in the experiments, which were approved by the Ethical Committee of the University Medical Center Göttingen (UMG). Before starting the tests, the subjects signed an informed consent form. The experimental session lasted approximately 1.5 h.
Tasks included in the offline assessment. The tasks combined square (S) and triangular (T) force profile, force levels (33 and 66 % MVC) of flexion (F) and extension (E), and two execution speed for each force profile
Cycle Length (sec)
We investigated the performance of the algorithms with full (192) and reduced number of channels (96, 48, 24, 16, 12, 10, 8, 6 and 4) where the channels were selected in the form of regular grids (see Fig. 6-e). The latter was chosen having in mind the future practical application of the methods, which should ideally, for the sake of simplicity, rely on the regular placement independent of the anatomy or the activity hot spots. The recorded data for each subject was split into training and testing sets (approximately 70 and 30 % of the whole dataset respectively, see Table 2): the tasks with only flexion or extension contractions were assigned to the training set (Table 2, Task 1–8), while the tasks with both flexion and extension were selected for the testing (Table 2, Task 9–12). The performance measures were common for online and offline tests and are described below (see section Performance measures).
Nine healthy male subjects (age between 23 and 38 years) were recruited for the online algorithm evaluation. Each subject signed an informed consent before commencing the experiment, which was approved by the Ethical Committee of the University Medical Center Göttingen (UMG). The same test was performed twice on consecutive days in order to evaluate the effect of training. The experimental session lasted maximum 2 h.
Training data collection
After the training, a test with online control was performed. The subject controlled 4 visual markers (blue circle in Fig. 3-c), each associated with the activation of one finger. The subjects now actively and proportionally controlled the vertical position of the control marker by increasing or decreasing the finger force in the direction of flexion (marker moving downward) or extension (marker moving upward). The vertical position of the marker (blue circle in Fig. 3-c) was determined by a normalized output of the tested control method (CSP-PE, LDA and THR), as explained in section Regression Methods. To quantify the control performance, the subjects performed online target tracking tasks, in which the subject tracked a moving reference marker (red circle in Fig. 3-c) by activating appropriate finger at the appropriate level. The aim was to maintain the smaller blue marker (Fig, 3-c), indicating the estimated activation level, within the larger red circle (Fig. 3-c), representing the desired activation. Whenever the controlled marker was within the reference circle, the color of the reference would turn into green. The task comprised 16 individual finger activations (4 for each finger) in which the reference marker moved vertically from 0 % MCV to 80 % MVC position, rested on the 80 % MVC level, and then returned back to the 0 % MVC value. The upward and downward movements of the reference marker lasted for 4 (slow ramp) or 2 (fast ramp) seconds, respectively, and the marker stayed at the plateau level for 3 s. Each finger was therefore activated two times in flexion and two times in extension and for both contractions one cycle was slow and the other fast. The fingers (reference markers) were activated in a random order. The test was repeated for each control method, also in a random order to avoid that the learning across methods influences the performance. The subjects were blinded as to which algorithm was under test in each session.
The estimated fingertip forces in the offline experiment and muscle activation levels in the online experiment were low-pass filtered using a moving average filter applied to 5 successive samples starting at each sample of the original signal. The Pearson correlation coefficient (PCORR) between the estimated and reference forces was computed to quantify the similarity in the signal shapes, and the mean square error normalized (nMSE) by the peak-to-peak value of reference force profile was calculated to assess the difference in signal amplitudes . As shown in Results section, the trends revealed by the two outcome measures, nMSE and PCORR, were equivalent over a comprehensive dataset collected and analyzed in the offline experiments. Therefore, the quality of the online tracking was evaluated by computing the nMSE only. In both the experiments, this analysis was performed for each finger during the segments of the reference trajectory in which that specific finger was supposed to be active (target finger in the task). The corresponding segments were named active phases. In order to evaluate the amount of false finger activations (i.e., finger estimated to be active when it should have been relaxed), the mean amplitude of the false activations (MAFA) outside of the respective active phases was calculated. The segments of the reference trajectory in which the finger was not supposed to be activated were named silent phases. The CA was evaluated calculating the overall success rate as the trace of the confusion matrix, divided by the total number of classified instances. Finally, the selectivity and specificity of the classifiers were calculated in one vs all configuration, where the median success rate of all the classes was used to compare the performance of the two methods for different electrode subsets.
The Kolmogorov-Smirnov test determined that the data were not normally distributed. Therefore, the data were statistically analyzed using non-parametric tests. To assess the statistically significant difference at the group level, the Friedman test was applied. If the Friedman test determined the difference, the conditions were compared pairwise using the Wilcoxon signed-rank tests with Bonferroni correction. A level of p < 0.05 was selected as the threshold for the statistical significance. In the offline experiment, the factors were number of channels (96, 48, 24, 16, 12, 10, 8, 6 and 4), and method (CSP-PE and LDA). In the online experiment, the factors were finger movements (IF, IE, MF, ME, RF, RE, LF, LE), and method (CSP-PE, LDA and THR). Bartlett multiple-sample test for equal variances was applied to determine statistically significant difference in dispersions within the conditions overall, followed by Ansari-Bradley two-sample test with Bonferroni correction for pairwise comparisons of the force variability between the conditions.
Offline finger force prediction
Percentage of selectivity and specificity across different electrode subset extracted from the HD-sEMG matrix. The values are reported as median and interquartile range (25–75 %)
CSP-PE Median and IQR (25–75 %)
LDA Median and IQR (25–75 %)
CSP-PE Median and IQR (25–75 %)
LDA Median and IQR (25–75 %)
Online control performance
There was a trend indicating an increase in performance across the two sessions, since the median values of the outcome measures improved, i.e., the nMSE and MAFA decreased with statistically significant differences (respectively, χ 2 = 221.2 p ≪0.001 and χ 2 = 163.7 p ≪0.001). The effect of training was least expressed when using the CSP-PE, which was characterized with a high level of accuracy in both sessions. For example, the median nMSE decreased for only ~1 % for CSP-PE (p ≪0.001) in the second session compared to the first, while in the case of LDA and THR, the median improvement in tracking accuracy was approximately 6 % between the two sessions (p ≪0.001). The results reported in this section refer to the second session of the experiment.
The results for the false activations complement those for the accuracy of tracking. The median MAFA was the highest for the THR consistently for all the movements, although there were no significant differences between the methods. However, some movements were controlled with substantially lower MAFA, for example, the MAFA for the index and middle finger extension was low with all three methods (e.g., compare to index flexion). Yet, the post hoc comparison did not reveal any significant differences between the specific methods across the movement classes.
The experiment was performed twice on consecutive days in order to evaluate the effect of training, and reducing the bias between subjects. Nevertheless, the lack of experience in EMG control may explain the overall variability between subjects showed in Fig. 8-c. For good control, the subject needs to activate each finger consistently, generating reproducible patterns of muscle activation that can be discriminated by the classifier/regressor. In addition, the patterns need to correspond to the ones generated during the training. The ability to reliably execute such patterns is likely subject dependent and two sessions were not enough for the consistency of control to improve and converge to a similar level across subjects. The figure shows the results for the accuracy of tracking (nMSE) for each subject as a radar graph, where each spoke represents one of the target finger movements and the length of the spokes was normalized to the maximum nMSE. This representation reveals participants who reached high level of accuracy with: i) all the three methods (s2 and s4), ii) the CSP-PE and the LDA (s1, s5, s6, s7 and s8), iii) the CSP-PE and the THR (s9) and finally iv) the CSP-PE only (s3). It is reasonable to expect that with practice, the online performance of the algorithms can be further improved, producing a more coherent behavior across participants . The initial uncertainty and variability of control and substantial improvement due to training are characteristic for myoelectric control in general . Regarding THR, for most subjects and movements the performance was similar to that of the CSP-PE and the LDA. However, as indicated by the few pronounced spikes in the radar plot, there were one or two specific movements in some subjects (see s1, s5, s7, and s8) that were substantially more difficult to control using THR. The performance dropped significantly in these few cases, decreasing the overall average accuracy and increasing the overall variability.
In the summary results shown in Fig. 8-b, the nMSE obtained with the CSP-PE, LDA and THR (median and IQR 25–75 %) was 3.6 % (2.5–5 %), 2.5 % (1.5–3.9 %) and 3.8 % (1.8–10.8 %), respectively. There was no statistically significant difference between the CSP-PE and the LDA, whereas the two machine learning methods were significantly different compared to the THR (p < 0.01). Furthermore, the CSP-PE and LDA exhibited similar dispersion, which was significantly lower compared to THR. The same trend holds for the MAFA. As shown in Fig. 8-b, the MAFA of the CSP-PE, LDA and THR was 0.7 % (0.2–2.2 %), 0.6 % (0.2–2.6) and 2.2 % (0.24–9.9 %), respectively. Both the CSP-PE and the LDA performed similarly, and they differed significantly with respect to THR (p < 0.01) both in median and dispersions.
The general similarity in the performance of the CSP-PE and the LDA was also confirmed by the CA (median and IQR 25–75 %), which were 91 % (87.7–91.7 %) for the CSP-PE and 90.3 % (89.4–93.25 %) for the LDA with no statistically significant differences.
The experiments demonstrated that the finger activation could be successfully decoded for different target activation profiles. Overall, the experiments demonstrated a more stable performance of the CSP-PE across the tested conditions. The CSP-PE exhibited less improvement between different sessions and outperformed the THR in online control and the LDA in offline tests. Furthermore, the study showed that a simple method, exploiting the topographical information about the individual finger activity from the HD-sEMG, provided in most cases regression accuracy similar to the pattern recognition techniques. However, THR lacked robustness in the sense that performance was not consistent across subjects and fingers.
Offline and online proportional control
In the offline experiment, the performance of the CSP-PE and the LDA increased with the number of channels, saturating to a stable level for more than 24 electrodes. Importantly, for less than 24 electrodes, the CSP-PE outperformed the LDA consistently in all outcome measures, except CA, yielding more accurate force estimates in active phases and better suppression of false activations in the silent phases. This confirmed the hypothesis that the mathematical properties of the CSP-PE, as described in , make this method especially effective in the context of selective finger activation. Increasing the number of channels evened out the performance of the two methods with respect to the quality of tracking in the active phase, but did not change the superiority of the CSP-PE in filtering out the activations during the silent phases.
The results of the online experiments were in accordance with the insights from the offline tests. The tracking accuracy was similar with both the CSP-PE and the LDA, with the median nMSE of approximately 3.6 and 2.6 % in online experiment, and 3.6 and 3.5 % respectively in offline experiment (48 electrodes). As explained in Methods, the online experiment resulted in more than 24 electrodes selected by the experimenter. According to the trends revealed offline (Fig. 6), this number of channels was enough to even out the performances of the two algorithms. In addition, during online tests the subjects could use visual feedback to adapt the activations during the trial, as demonstrated in .
The CSP-PE exhibited more stable accuracy across the tested conditions. The performance was good from the beginning and similar across the two successive sessions, providing the least improvement in outcome measures. Furthermore, the CSP-PE produced better performance for lower number of channels (Fig. 6) compared to LDA. This means that the CSP-PE might be less sensitive to the subjective factors than the LDA, which produced similar accuracy but only for the high number of channels and after a session of practice.
In the offline experiment, the electrodes were selected as a regular grid, without any relation to the specific finger activation patterns. Regular electrode grids are convenient for practical implementation and allow simple mounting, since they can be realized as extensible uniform bracelets that are simply wrapped around the forearm . Future investigations will further address the minimization of the number of channels, determining acceptable electrode locations and optimizing electrode-recording configurations, using the established methods for feature reduction [57–59]. Importantly, the results of the present study (offline tests) demonstrate that the number of electrodes can be decreased substantially (from 192 to 10) without significantly compromising the performance. This is an optimistic result implying that the proposed methods could be translated into the clinical context using multichannel EMG braces comprising practical dry electrodes.
Thresholding for online control
The online experiment demonstrated the feasibility of simple thresholding for proportional control of individual fingers, albeit with some limitations. Exploiting a dense array of detection points provided by HD-sEMG interface, distinct areas of focal sEMG activity could be identified for each finger , with an overlap in some cases due to anatomical constraints and crosstalk . When the areas overlapped, the experimenter did not select the electrodes from the intersection. The THR method has a low computational cost and there is no training; after the electrode selection, the experimenter provided only a fast calibration of the thresholds associated with each finger movement. However, the THR method also exhibited some drawbacks. In the second session, three out of nine subjects were not able to activate a certain finger movement, due to a significant overlap, i.e., the activity of finger involved in the task projected strongly to neighboring areas triggering thereby other fingers. The THR was also less successful in suppressing the false activations. Nevertheless, when the subjects were able to control the finger, the tracking accuracy was actually comparable to the performances of the two machine learning approaches. Therefore, the THR is not universally applicable. However, it can be used successfully in some subjects or with a reduced number of movement classes, controlling only those fingers (or finger groups) characterized with distinct and separate areas of activity.
Application for rehabilitation
The aforementioned conclusions could provide useful guidelines for the translation of the tested methods to the clinical context, targeting dexterous control of hand rehabilitation robots. For example, stroke patients have impaired motor functions characterized with pathological synergies. At the hand level, this is expressed as a difficulty in selectively activating individual fingers . The methods developed in the present study could be applied to implement a user-driven control of a rehabilitation system. For example, the activation signals estimated for each finger, as demonstrated in the online tests, could be used as the control signals in the isometric mode of Amadeo, i.e., to implement the functionality of the force joystick, as when playing simple video games by producing appropriate isometric forces. More generally, the estimated signals could be used to trigger and/or modulate the movement of the Amadeo finger motors (dynamic control) . Since the Amadeo system offers a set of therapeutic individual finger exercises, the future perspective is to integrate the individual finger myocontrol developed in the present study with these motivational tasks, creating thereby an innovative, engaging and user-responsive training program. Considering the future clinical application and following the results of the offline analysis, the control could be implemented with substantially less electrodes, which would allow using a practical EMG bracelet such as Myoband  combined with the CSP-PE. Alternatively, even a full HD-sEMG electrode system could be used practically if integrated in a textile garment (e.g., ). Importantly, for online control of Amadeo, the arm/hand will be supported exactly as in the present study, and it is therefore not required to train/test the algorithms with the arm in different positions, as usual in prosthetics to increase the robustness of the classification/regression across arm postures [61, 62].
Another potential application is the extension of the methods to the control of dexterous hand prostheses. At the current level, the methods tested in the present study could not be directly translated for general prosthetic application, as the fingers are controlled sequentially, one by one. However, such a controller could be used to support some specific functions (e.g., typing on a keyboard) exploiting the individual finger actuation available in the modern prostheses (e.g. i-Limb ). For example, keyboard typing could be implemented through classification (on-off), or the estimated force could be mapped to the finger velocity, allowing proportional control of the speed of finger flexion/extension (instead of force). The latter is not essential for typing, but it could allow the subject to type faster as he/she becomes more trained and skilled in control. More importantly, the present study demonstrates the feasibility of achieving fine and selective control of individual fingers, across a comprehensive number of tasks (force profiles) and with a reduced set of electrodes. Nevertheless, the control of a prosthesis requires a more natural and functional approach, and the future work will be to study the simultaneous regression of multiple fingers (including the thumb) using HD-sEMG setup.
The translation into the clinical context faces a number of challenges, which will be addressed in future work. For example, as explained above, stroke patients can have significant reduction in muscle forces and impaired coordinative control. Due to weak activity and pathological synergies , the activation patterns for the individual finger movements are likely to be significantly less discriminative and thereby more difficult to classify and estimate. In this context, nevertheless, an adaptive training can be envisioned in which the patient and the system coadapt  and evolve through the process of recovery. Initially, the system can estimate a subset of movements, limited to those that can be well discriminated. This can be used to start the training, promoting the initial recovery, and when the activity maps become better differentiated, new movements can be included.
Finally, as already pointed out in myoelectric control for prosthetics , it would be of interest for clinical applications to minimize the time and effort (subjects and staff) need for the training. Ideally, the training/calibration should be short and easy and without the need for frequent retraining. In the present study, we demonstrated that a reasonable quality of control can be achieved with an in-session training. The training is especially simple and easy to understand in the case of THR, since it reduces to selecting the channels with strong activation in a color map and then visually adjusting the thresholds (average time 10 min in THR vs. 20 min for LDA/CSP-PE). Therefore, the THR might also allow for an easy retraining across sessions. However, in the ideal case, the retraining would not be necessary. This was not tested in the present study but it is certainly an important future goal. The robustness of the methods in terms of retraining could be assessed by testing the control across sessions using the same, previously collected data (no retraining). In this context, the use of HD-sEMG interface might be particularly beneficial as an increased resolution contributes with redundant information, and this can be used to increase robustness. An illustrative demonstration is provided in  by extracting features reducing the sensitivity to electrodes shifts.
The present study investigated methods for selective estimation of individual finger movements, motivated by the final aim of implementing an online protocol for dexterous finger control using a hand rehabilitation robot. We detected the intention to move a single finger from electromyographic signals providing proportional control while reducing the simultaneous co-activations of other fingers during both offline and online experiments. The insights from the present study can be used to guide the implementation of a practical myoelectric system for dexterous control in hand rehabilitation robotics and prosthetics. More specifically, the results demonstrated that despite the abundance of detection points in HD-sEMG, a simple method based on thresholding (THR) exhibited serious drawbacks, and that therefore the pattern recognition is still the method of choice for robust practical implementations. Next, provided that the recording is above a certain resolution (>24 channels), either of the pattern recognition methods (CSP-PE and LDA) can be selected to implement the control. In this case, information redundancy compensates for the favorable mathematical properties of the CSP-PE vs. LDA. Finally, if only a reduced number of electrodes is available (≤12), the CPS-PE is the recommended approach.
CA, classification accuracy; CSP-PE, common spatial pattern proportional estimator; DIP, distal interphalangeal; HD-sEMG, high density surface electromyography; IE, index extension; IF, index flexion; IQR, interquartile range; LDA, linear discriminant analysis; LE, little extension; LF, little flexion; MAFA, mean amplitude of the false activations; MCP, metacarpophalangeal; ME, middle extension; MF, middle flexion; MVC, maximum voluntary contraction; nMSE, normalized mean square error; PCORR, Pearson correlation coefficient; PIP, proximal interphalangeal; RE, ring extension; RF, ring flexion; RMS, root mean square; THR, thresholding
The authors would like to acknowledge Andrei Ninu for his help in designing experiments, preparing the experimental setup and collecting the data.
This research was supported by the European Commission under the MYOSENS (FP7-PEOPLE-2011-IAPP-286208) project and by the German Ministry for Education and Research (BMBF) via the Bernstein Focus Neurotechnology (BFNT) Gottingen under Grant 01GQ0810.
Availability of data and materials
Data will not be shared at this stage, because of a large volume of data and online storage restrictions.
DF, SD, IB, PA and NC conceived and designed the experiments. NC and SD implemented the protocol and conducted the experiments. NC analyzed the data. DF, SD, PA and NC participated in writing the manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Each subject signed an informed consent form before commencing the experiment, which was approved by the Ethical Committee of the University Medical Center Göttingen (document: 32/2/16).
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Schieber MH, Poliakov AV. Partial inactivation of the primary motor cortex hand area: effects on individuated finger movements. J Neurosci. 1998;18:9038–54.PubMedGoogle Scholar
- Oby ER, Ethier C, Miller LE. Movement representation in the primary motor cortex and its contribution to generalizable EMG predictions. J Neurophysiol. 2013;109:666–78.View ArticlePubMedGoogle Scholar
- Asghari Oskoei M, Hu H. Myoelectric control systems—a survey biomed. Signal Process Control. 2007;2:275–94.View ArticleGoogle Scholar
- Fougner A, Stavdahl O, Kyberd PJ, Losier YG, Parker PA. Control of upper limb prostheses: terminology and proportional myoelectric control-a review. IEEE Trans Neural Syst Rehabil Eng. 2012;20:663–77.View ArticlePubMedGoogle Scholar
- Nielsen JLG, Holmgaard S, Jiang N, Englehart KB, Farina D, Parker PA. Simultaneous and proportional force estimation for multifunction myoelectric prostheses using mirrored bilateral training. IEEE Trans Biomed Eng. 2011;58(3 PART 1):681–8.View ArticlePubMedGoogle Scholar
- Muceli S, Member S, Farina D, Member S. Simultaneous and proportional estimation of hand kinematics from EMG during mirrored movements at multiple degrees-of-freedom. IEEE Trans Neural Syst Rehabil Eng. 2012;20:371–8.View ArticlePubMedGoogle Scholar
- Jiang N, Englehart KB, Parker PA. Extracting simultaneous and proportional neural control information for multiple-DOF prostheses from the surface electromyographic signal. IEEE Trans Biomed Eng. 2009;56:1070–80.View ArticlePubMedGoogle Scholar
- Amsuess S, Gobel P, Graimann B, Farina D. A multi-class proportional myocontrol algorithm for upper limb prosthesis control: validation in real-life scenarios on amputees. IEEE Trans neural Syst Rehabil Eng. 2014;4320(c):1–11.Google Scholar
- Cipriani C, Zaccone F, Micera S, Carrozza MC. On the shared control of an EMG-controlled prosthetic hand: analysis of user–prosthesis interaction. IEEE Trans Robot. 2008;24:170–84.View ArticleGoogle Scholar
- Belter JT, Segil JL, Dollar AM, Weir RF. Mechanical design and performance specifications of anthropomorphic prosthetic hands: A review. J Rehabil Res Dev. 2013;50:599.View ArticlePubMedGoogle Scholar
- Chiri A, Vitiello N, Giovacchini F, Roccella S, Vecchi F, Carrozza MC. Mechatronic design and characterization of the index finger module of a hand exoskeleton for post-stroke rehabilitation. IEEE/ASME Trans Mechatronics. 2012;17:884–94.View ArticleGoogle Scholar
- Tong KY, Member S, Hu XL, Fung KL, Wei XJ, Rong W, Susanto EA. An EMG-driven exoskeleton hand robotic training device on chronic stroke subjects task training system for stroke rehabilitation. IEEE Int Conf Rehabil Robot. 2011;2011:5975340.PubMedGoogle Scholar
- Wege A, Kondak K, Hommel G. Force control strategy for a hand exoskeleton based on sliding mode position control. IEEE/RSJ Int Conf Intell Robot Syst. 2006:4615–4620.Google Scholar
- Zhang X, Zhou P. High-density myoelectric pattern recognition. IEEE Trans Biomed Eng. 2012;59:1649–57.View ArticlePubMedGoogle Scholar
- Al-Timemy AH, Bugmann G, Escudero J, Outram N. Classification of finger movements for the dexterous hand prosthesis control with surface electromyography. IEEE J Biomed Heal Informatics. 2013;17:608–18.View ArticleGoogle Scholar
- Merletti R, Parker PA. Electromyography: Physiology, Engineering, and Non-Invasive Applications. 2004.Google Scholar
- Shrirao N a, Reddy NP, Kosuri DR. Neural network committees for finger joint angle estimation from surface EMG signals. Biomed Eng Online. 2009;8:2.View ArticlePubMedPubMed CentralGoogle Scholar
- Antfolk C, Cipriani C, Controzzi M, Carrozza MC, Sebelius F. Using EMG for real-time prediction of joint angles to control a prosthetic hand equipped with a sensory feedback system. J Med Biol Eng. 2010, 30:399–406.Google Scholar
- Hioki M, Kawasaki H. Estimation of Finger Joint Angles from sEMG Using a Neural. ISRN Rehabil. 2012, 2012.Google Scholar
- Ngeo JG, Tamei T, Shibata T. Continuous and simultaneous estimation of finger kinematics using inputs from an EMG-to-muscle activation model. J Neuroeng Rehabil. 2014;11:1–14.View ArticleGoogle Scholar
- Castellini C, Koiva R. Using surface electromyography to predict single finger forces. 2012 4th IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatronics. 2012:1266–1272.Google Scholar
- Gijsberts A, Bohra R, Sierra González D, Werner A, Nowak M, Caputo B, Roa M a, Castellini C. Stable myoelectric control of a hand prosthesis using non-linear incremental learning. Front Neurorobot. 2014;8(February):8.PubMedPubMed CentralGoogle Scholar
- Ravindra V, Castellini C. A comparative analysis of three Non-invasive human-machine interfaces for the disabled. Front Neurorobot. 2014;8(October):1–10.Google Scholar
- Taub E, Uswatte G, Elbert T. New treatments in neurorehabilitation founded on basic research. Nat Rev Neurosci. 2002;3:228–36.View ArticlePubMedGoogle Scholar
- Biitefisch C, Hummelsheim H, Denzler P. Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. J Neurol Sci. 1995;130:59–68.View ArticleGoogle Scholar
- Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet. 2011;377:1693–702.View ArticlePubMedGoogle Scholar
- Johansson BB. Current trends in stroke rehabilitation. A review with focus on brain plasticity. Acta Neurol Scand. 2011;123:147–59.View ArticlePubMedGoogle Scholar
- Merians AS, Poizner H, Boian R, Burdea G, Adamovich S. Sensorimotor training in a virtual reality environment: does it improve functional recovery poststroke? Neurorehabil. Neural Repair. 2006;20:252–67.View ArticleGoogle Scholar
- Norouzi-Gheidari N, Archambault PS, Fung J. Effects of robot-assisted therapy on stroke rehabilitation in upper limbs: systematic review and meta-analysis of the literature. J Rehabil Res Dev. 2012;49:479–96.View ArticlePubMedGoogle Scholar
- Colombo R, Pisano F, Micera S, Mazzone A, Delconte C, Carrozza MC, Dario P, Minuco G. Robotic techniques for upper limb evaluation and rehabilitation of stroke patients. IEEE Trans Neural Syst Rehabil Eng. 2005;13:311–24.View ArticlePubMedGoogle Scholar
- Mulas M, Folgheraiter M, Gini G. An EMG-Controlled Exoskeleton for Hand Rehabilitation. 9th Int. Conf. Rehabil. Robot. 2005. ICORR 2005. 2005:371–374.Google Scholar
- Dipietro L, Ferraro M, Palazzolo JJ, Krebs HI, Volpe BT, Hogan N. Customized interactive robotic treatment for stroke: EMG-triggered therapy. IEEE Trans Neural Syst Rehabil Eng. 2005;13:325–34.View ArticlePubMedPubMed CentralGoogle Scholar
- Prange GB, Jannink MJ a, Groothuis-Oudshoorn CGM, Hermens HJ, IJzerman MJ. Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke. J Rehabil Res Dev. 2006;43:171.View ArticlePubMedGoogle Scholar
- Schieber MH, Lang CE, Reilly KT, McNulty P, Sirigu A. Selective activation of human finger muscles after stroke or amputation. Adv Exp Med Biol. 2009;629:559–75.View ArticlePubMedPubMed CentralGoogle Scholar
- Leijnse JN a L, Campbell-Kyureghyan NH, Spektor D, Quesada PM. Assessment of individual finger muscle activity in the extensor digitorum communis by surface EMG. J Neurophysiol. 2008;100:3225–35.View ArticlePubMedGoogle Scholar
- Gallina A, Botter A. Spatial localization of electromyographic amplitude distributions associated to the activation of dorsal forearm muscles. Front Physiol. 2013;4(December):367.PubMedPubMed CentralGoogle Scholar
- Gazzoni M, Celadon N, Mastrapasqua D, Paleari M, Margaria V, Ariano P. Quantifying forearm muscle activity during wrist and finger movements by means of multi-channel electromyography. PLoS One. 2014;9:e109943.View ArticlePubMedPubMed CentralGoogle Scholar
- Peleg D, Braiman E, Yom-Tov E, Inbar GF. Classification of finger activation for use in a robotic prosthesis arm. IEEE Trans Neural Syst Rehabil Eng. 2002;10:290–3.View ArticlePubMedGoogle Scholar
- Tenore FVG, Ramos A, Fahmy A, Acharya S, Etienne-Cummings R, Thakor NV. Decoding of individuated finger movements using surface electromyography. IEEE Trans Biomed Eng. 2009;56:1427–34.View ArticlePubMedGoogle Scholar
- Jiang N, Rehbaum H, Vujaklija I, Graimann B, Farina D. Intuitive, online, simultaneous, and proportional myoelectric control over two degrees-of-freedom in upper limb amputees. IEEE Trans Neural Syst Rehabil Eng. 2014;22:501–10.Google Scholar
- Gijsberts A, Atzori M, Castellini C, Muller H, Caputo B. Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification. IEEE Trans Neural Syst Rehabil Eng. 2014;22:735–44.View ArticlePubMedGoogle Scholar
- Birdwell JA, Hargrove LJ, Weir RF, Kuiken T a. Extrinsic finger and thumb muscles command a virtual hand to allow individual finger and grasp control. IEEE Trans Biomed Eng. 2015;62:218–26.View ArticlePubMedGoogle Scholar
- Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003;50:848–54.View ArticlePubMedGoogle Scholar
- Englehart K, Hudgins B, Parker PA, Member S. A wavelet-based continuous classification scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2001;48:302–11.View ArticlePubMedGoogle Scholar
- Fukunaga K. Introduction to Statistical Pattern Recognition. 2013.Google Scholar
- Scheme EJ, Member S, Englehart KB, Member S, Hudgins BS, Member S. Selective classification for improved robustness of myoelectric control under nonideal conditions. IEEE Trans Biomed Eng. 2011;58:1698–705.View ArticlePubMedGoogle Scholar
- Parker P, Englehart K, Hudgins B. Myoelectric signal processing for control of powered limb prostheses. J Electromyogr Kinesiol. 2006;16:541–8.View ArticlePubMedGoogle Scholar
- Hahne JM, Graimann B, Muller KR. Spatial filtering for robust myoelectric control. IEEE Trans Biomed Eng. 2012;59:1436–43.View ArticlePubMedGoogle Scholar
- Tomioka R, Lemm S. Filters for Robust EEG. IEEE Signal Process Mag. 2008;2008:41–56.Google Scholar
- Ramoser H, Müller-Gerking J, Pfurtscheller G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng. 2000;8:441–6.View ArticlePubMedGoogle Scholar
- Blankertz B, Dornhege G, Krauledat M, Müller K-R, Curio G. The non-invasive Berlin Brain-Computer Interface: fast acquisition of effective performance in untrained subjects. Neuroimage. 2007;37:539–50.View ArticlePubMedGoogle Scholar
- Lorrain T, Jiang N, Farina D. Influence of the training set on the accuracy of surface EMG classification in dynamic contractions for the control of multifunction prostheses. J Neuroeng Rehabil. 2011;8:25.View ArticlePubMedPubMed CentralGoogle Scholar
- Lang CE. Human finger independence: limitations due to passive mechanical coupling versus active neuromuscular control. J Neurophysiol. 2004;92:2802–10.View ArticlePubMedGoogle Scholar
- Jiang N, Vujaklija I, Rehbaum H, Member S, Graimann B. Is accurate mapping of EMG signals on kinematics needed for precise online myoelectric control ? IEEE Trans Neural Syst Rehabil Eng. 2014;22:549–58.View ArticlePubMedGoogle Scholar
- Radhakrishnan SM, Baker SN, Jackson A. Learning a novel myoelectric-controlled interface task. J Neurophysiol. 2008;100:2397–408.View ArticlePubMedPubMed CentralGoogle Scholar
- Myo armband. [https://www.thalmic.com/]. Accessed 12 May 2016.
- Geng Y, Zhang X, Zhang Y-T, Li G. A novel channel selection method for multiple motion classification using high-density electromyography. Biomed Eng Online. 2014;13:102.View ArticlePubMedPubMed CentralGoogle Scholar
- Huang H, Zhou P, Li G, Kuiken T. Spatial filtering improves EMG classification accuracy following targeted muscle reinnervation. Ann Biomed Eng. 2009;37:1849–57.View ArticlePubMedPubMed CentralGoogle Scholar
- Hwang HJ, Hahne JM, Müller KR. Channel selection for simultaneous myoelectric prosthesis control. 2014 Int. Winter Work. Brain-Computer Interface. BCI. 2014;2014:056008.Google Scholar
- “Amadeo solution”, Tyromotion GmbH [http://tyromotion.com/wp-content/uploads/2013/04/HartwigM-2014-The-Tyrosolution-Concept._EN.pdf]. Accessed 12 May 2016.
- Geng Y, Zhou P, Li G. Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees. J Neuroeng Rehabil. 2012;9(74):1–11.Google Scholar
- Fougner A, Scheme E, Member S, Chan ADC, Member S, Englehart K, Member S, Stavdahl Ø. Resolving the limb position effect in myoelectric pattern recognition. IEEE Trans Neural Syst Rehabil Eng. 2011;19:644–51.View ArticlePubMedGoogle Scholar
- “i-Limb Ultra: User Manual” Touch Bionics Ltd. [http://www.touchbionics.com/sites/default/files/files/i-limb ultra revolution user manual September 2014.pdf]. Accessed 12 May 2016.
- Li X, Liu J, Li S, Wang Y. Examination of hand muscle activation and motor unit indices derived from surface EMG in chronic stroke. IEEE Trans Biomed Eng. 2014;61:2891–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Blank AA, French JA, Pehlivan AU, O’Malley MK. Current trends in robot-assisted upper-limb stroke rehabilitation: promoting patient engagement in therapy. Curr Phys Med Rehabil Reports. 2014;2:184–95.View ArticleGoogle Scholar
- Farina D, Member S, Jiang N, Rehbaum H, Member S, Holobar A, Graimann B, Dietl H, Aszmann OC. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans Neural Syst Rehabil Eng. 2014;22:797–809.View ArticlePubMedGoogle Scholar
- Stango A, Member S, Negro F, Farina D, Member S. Spatial correlation of high density EMG signals provides features robust to electrode number and shift in pattern recognition for myocontrol. IEEE Trans Neural Syst Rehabil Eng. 2015;23:189–98.View ArticlePubMedGoogle Scholar
- You K-J, Rhee K-W, Shin H-C. Finger motion decoding using EMG signals corresponding various arm postures. Exp Neurobiol. 2010;19:54–61.View ArticlePubMedPubMed CentralGoogle Scholar
- Cipriani C, Antfolk C, Controzzi M, Lundborg G, Rosen B, Carrozza MC, Sebelius F. Online myoelectric control of a dexterous hand prosthesis by transradial amputees. IEEE Trans Neural Syst Rehabil Eng. 2011;19:260–70.View ArticlePubMedGoogle Scholar
- Khushaba RN, Kodagoda S, Takruri M, Dissanayake G. Toward improved control of prosthetic fingers using surface electromyogram (EMG) signals. Expert Syst Appl. 2012;39:10731–8.View ArticleGoogle Scholar
- Sebelius F, Eriksson L, Balkenius C, Laurell T. Myoelectric control of a computer animated hand: a new concept based on the combined use of a tree-structured artificial neural network and a data glove. J Med Eng Technol. 2006;30:2–10.View ArticlePubMedGoogle Scholar