Skip to main content

A compact system for simultaneous stimulation and recording for closed-loop myoelectric control

Abstract

Background

Despite important advancements in control and mechatronics of myoelectric prostheses, the communication between the user and his/her bionic limb is still unidirectional, as these systems do not provide somatosensory feedback. Electrotactile stimulation is an attractive technology to close the control loop since it allows flexible modulation of multiple parameters and compact interface design via multi-pad electrodes. However, the stimulation interferes with the recording of myoelectric signals and this can be detrimental to control.

Methods

We present a novel compact solution for simultaneous recording and stimulation through dynamic blanking of stimulation artefacts. To test the system, a feedback coding scheme communicating wrist rotation and hand aperture was developed specifically to stress the myoelectric control while still providing meaningful information to the subjects. Ten subjects participated in an experiment, where the quality of closed-loop myoelectric control was assessed by controlling a cursor in a two degrees of freedom target-reaching task. The benchmark performance with visual feedback was compared to that achieved by combining visual feedback and electrotactile stimulation as well as by using electrotactile feedback only.

Results

There was no significant difference in performance between visual and combined feedback condition with regards to successfully reached targets, time to reach a target, path efficiency and the number of overshoots. Therefore, the quality of myoelectric control was preserved in spite of the stimulation. As expected, the tactile condition was significantly poorer in completion rate (100/4% and 78/25% for combined and tactile condition, respectively) and time to reach a target (9/2 s and 13/4 s for combined and tactile condition, respectively). However, the performance in the tactile condition was still good, with no significant difference in path efficiency (38/8%) and the number of overshoots (0.5/0.4 overshoots), indicating that the stimulation was meaningful for the subjects and useful for closed-loop control.

Conclusions

Overall, the results demonstrated that the developed system can provide robust closed-loop control using electrotactile stimulation. The system supports different encoding schemes and allows placing the recording and stimulation electrodes next to each other. This is an important step towards an integrated solution where the developed unit will be embedded into a prosthetic socket.

Background

Even though myoelectric prostheses can partially restore lost functionality following transradial amputations or congenital limb deficiency, approximately 25% of the users abandon their prosthesis [1]. In addition to the requirements related to ergonomics and control [2], users also report the restoration of exteroceptive and proprioceptive feedback as an important future goal [3]. In the absence of somatosensory information, the users need to rely on visual and/or auditory input, which can provide a good estimate of the prosthesis state [4], but requires the subject to direct the attention to the prosthesis. A functional prosthesis without feedback can be viewed as an alien object; a mechanical tool required to accomplish the task rather than a bionic replacement of the missing limb. In fact, a sensory intact residual limb is in many cases more valuable to the user [5]. By closing the loop, it is expected that the users can reach a higher embodiment of the prosthesis since the need for visual attention will be reduced [6, 7]. Although demonstrating the functional benefits of feedback is not an easy task [8], several recent studies have indeed shown that providing sensory feedback can improve user performance [9,10,11] as well as experience [12, 13]. Currently, only three commercial prostheses promise to deliver non-invasive tactile feedback, namely, Vincent Evolution 4 (Vincent Systems, Karlsruhe, Germany) [14], Luke Arm (Mobius Bionics, Manchester, USA) [15] and Ability Hand (Psyonic, Champaign, USA) [16]; however, the benefits of feedback in these devices are yet to be clinically proven.

The feedback can be delivered either invasively or on the surface of the skin through various modalities [5, 17,18,19]. Invasive methods based on electrical stimulation of peripheral nerves [9, 20], spinal cord [21] and/or brain [22] can evoke somatotopic sensations, but they also demand a surgical procedure, and amputees can be reluctant to undergo additional invasive interventions [23]. A non-invasive approach is substitution feedback, which transmits the missing sensory information through another modality applied to a different site of the skin. Vibrotactile stimulation [24] utilizes small vibration motors to activate mechanoreceptors in the skin, whereas electrotactile stimulation [25] activates superficial nerves in the skin directly via surface electrodes. Although electrical stimulation can produce uncomfortable sensations if the parameters are not adjusted properly, it holds several advantages including low power consumption, independent parameter modulation and compact as well as flexible design [26, 27]. This can be particularly advantageous when using electrode arrays to present multiple feedback variables intuitively to the user [28].

An important drawback, however, when using electrotactile stimulation in closed-loop myoelectric control is that it interferes with the recorded myoelectric signal. The stimulation produces strong artifacts that can corrupt the signal and possibly saturate the electromyography (EMG) amplifier, leading to problems in prosthesis control.

The suppression of stimulation artifacts is a known and longstanding problem in the field of functional electrical stimulation (FES). In FES, the electrical stimulation is delivered to activate muscles and restore movements in paralyzed patients, and it can be combined with EMG to trigger the stimulation by detecting subject movement intention. The influence of spatial filters, electrode distance, and stimulation waveforms on the stimulation artifacts were evaluated in [29]. Software [30] and hardware blanking [31] as well as adaptive methods have been tested for artifact suppression [32, 33]. A specific challenge in FES is that the artifact also includes muscle electrical activity (M-wave) that is evoked by the stimulation itself and needs to be discriminated from the voluntary EMG.

Contrary to FES, the electrical stimulation in prosthetics is used to produce tactile sensations, which means that the stimulation is delivered below the motor threshold, hence no M-wave will be generated. Moreover, the pulse amplitude is lower and the current flow more localized, leading to artefacts of smaller amplitude. Nevertheless, the myoelectric control is also more sophisticated and relies on multichannel EMG and machine learning (classification and regression) to discriminate different movement patterns [34]. Several recent studies that employed electrotactile feedback avoided the interference by placing the recording and stimulation interface contralaterally [28, 35] or ipsilaterally but with large spatial separation (e.g., recording from the forearm while stimulating the upper arm [36]). This allows testing in laboratory but it is not applicable clinically, where both recording and stimulation have to be fitted within a confined area of the residual limb and prosthetic socket. This problem is exacerbated by the fact that both interfaces often include multiple channels (electrode arrays or matrices).

Only few studies have explicitly investigated the effect of electrotactile stimulation on myoelectric control. The presence of stimulation artefacts has shown to dramatically decrease the control performance when using an LDA classifier [37] and neural networks [38]. Delivering electrical stimulation more than 60 mm from the myoelectric recording site has shown not to interfere significantly with the control system [39]. Time-division multiplexing [40] and software blanking of stimulation artefacts [37] have been applied to allow simultaneous recording and stimulation without spatial separation between the electrodes. The former method divides stimulation time into on and off intervals (i.e., stimulation and recording windows), while continuously acquiring myoelectric signals. The contaminated signal (stimulation window) is then discarded and the artefact-free signal (recording window) is used for myoelectric control. While providing stable control, this approach does not allow a constant flow of feedback. Artefact blanking is a more selective approach, where the recording samples affected by an artefact after each electrical pulse are set to zero. For myoelectric control, the incoming EMG data are segmented for feature extraction [41]. In [37], various data segmentation strategies were applied to evaluate the classification accuracy of software blanked data sets. Even though the evaluation was performed offline, the same principles can be introduced in an online application by using a single device that integrates both a recording and stimulation unit.

The present study describes a novel system specifically designed for simultaneous electrotactile stimulation (16 channels) and myoelectric control (8 channels) with compact form factor (henceforth referred to as the MaxSens system). The system is described in detail, including hardware components and control algorithm with integrated dynamic blanking, and it was evaluated experimentally by assessing the quality of online myoelectric control with and without concomitant electrotactile stimulation. The electrotactile feedback was designed to include simultaneous modulation of multiple stimulation parameters to maximally “stress” the recording and classification system. The experimental evaluation confirmed successful application: online control was not compromised when activating stimulation and the tactile feedback was useful for the subjects when vision was unavailable. Therefore, the system recorded EMG of good quality while generating stimulation that the subjects could perceive and exploit for closed-loop control. These are encouraging results regarding the prospect of embedding the developed compact system in a myoelectric prosthesis.

System description

Electronic design

The MaxSens system is based on the concept of spatio-temporal distribution of stimulation pulses that was introduced for neurorehabilitation applications [42], and more recently proposed as an integrated and flexible solution for providing electrotactile feedback in the MaxSens stimulator prototype [27]. Further development of the system involved miniaturization of the device and adding features that resulted in a first standalone electrotactile closed-loop system that enables simultaneous recording and stimulation. The current system was briefly presented in a short conference publication [43]. The architecture of the system is shown in Fig. 1. As illustrated in this figure, the device consists of several independent units: microcontroller, electrical stimulation, EMG recording, communication and power supply.

Fig. 1
figure1

Block diagram of the units comprising the MaxSens stimulation and recording system (i.e., microcontroller, electrical stimulation, EMG recording, communication and power supply unit)

Microcontroller unit

The microcontroller unit (MCU) is the central part of the MaxSens system. The MCU controls and synchronizes the other components of the device. It is based on an 8-bit RISC microcontroller ATxmega384C3 (Microchip Technology, Arizona, USA) integrating a rich set of hardware peripherals and memory resources. The MCU operating clock frequency is 32 MHz. The connection between the MCU and the electrical stimulation unit is implemented using a serial peripheral interface (SPI) communication module and two compare modules. The SPI interface is used for accessing the digital/analogue (DA) converter in order to set the amplitude of the current pulses and control the high-voltage switches to select active cathode fields on a 16-channel matrix electrode (see below). The two compare modules (OC0x) equipped with two 16-bit timers control the current source (H-bridge) to produce biphasic current pulses.

The connection between the MCU and the EMG recording unit is implemented using an SPI module and general-purpose input/output pins (GPIO). The SPI module is used for setting the desired parameters (sample frequency and/or EMG gain) of the EMG unit and for reading recorded EMG samples. The GPIOs control the timing of the sampling process. The connection between the MCU and the communication unit is implemented using two USART modules. Through these modules, the MCU communicates with the USB and Bluetooth controllers to simultaneously receive stimulation commands and transmit recorded EMG samples to the host computer.

Electrical stimulation

The electrical stimulation unit is implemented as a current controlled H-bridge in order to produce biphasic current pulses. The H-bridge acts as a single channel stimulator generating a sequence of current pulses that are then distributed to matrix electrodes through 16-channel analog switches MAX14082 (Maxim Integrated, California, USA). The amplitudes of the current pulses are controlled by a 16-bit DA converter, while duration and frequency are set using the MCU compare modules and timers. The pulse width and amplitude can be adjusted for individual electrode pads in a range of 50–1000 µs with 10 µs steps and 50–10,000 µA with 0.1 µA steps, respectively. The stimulation frequency can be modulated in a 1–400 Hz range with 1 Hz steps and is common for all electrode pads. In each stimulation period, the pulses are generated sequentially and then distributed with adjustable inter-pulse interval (1–25 ms in steps of 1 ms) to all active stimulation channels, implementing a so-called asynchronous stimulation protocol. An illustration of the stimulation parameters and the asynchronous protocol in the case of two-channels is shown in Fig. 2. This approach is convenient for implementation, but it also enforces some constraints on the allowed combination of parameter values. Specifically, all pulses that should be distributed to active channels within a single period need to “fit” within that time interval. For instance, if we assume that all pulses have the same width, the following inequality must hold: \({\mathrm{N}}_{\mathrm{ch}}\cdot \left(2 \cdot {\mathrm{T}}_{\mathrm{pulse}}+ {\mathrm{T}}_{\mathrm{TBP}}\right)\le \frac{1000}{{\mathrm{F}}_{\mathrm{stim}}}\), where \({\mathrm{N}}_{\mathrm{ch}}\) is the number of active channels, \({\mathrm{T}}_{\mathrm{pulse}}\) and \({\mathrm{T}}_{\mathrm{TBP}}\) are pulse width and time between pulses in ms, respectively, and \({\mathrm{F}}_{\mathrm{stim}}\) is pulse frequency in Hz. For example, for \({\mathrm{T}}_{\mathrm{pulse}}\) = 0.5 ms, \({\mathrm{T}}_{\mathrm{TBP}}\) = 1 ms, and \({\mathrm{F}}_{\mathrm{stim}}\) = 100 Hz, maximally five channels can be simultaneously active.

Fig. 2
figure2

Illustration of the asynchronous stimulation protocol for a two-channel stimulation scenario, where IPI, F, P, PW, A and CH indicate inter-pulse interval, frequency, pulse, pulse width, amplitude and stimulation channel, respectively. The H-bridge outputs a multiplexed signal that is demultiplexed to the respective stimulation channels. All paremeters (IPI, F, PW and A) are equal before and after demultiplexing. The horizontal lines represents time, where the dotted horizontal segments indicate prolonged duration. The green and blue color indicate the blanking intervals, where the green color indicates blanking duration calculated from Eq. 1 or the first term within the brackets of Eq. 2 and the blue color indicates blanking duration calculated from the second term within the brackets of Eq. 2

EMG recording

The EMG unit is based on an integrated circuit (IC) ADS1299 (Texas Instruments, Texas, USA), which consists of eight differential 24-bit sigma delta analogue-to-digital converters with integrated programmable gain amplifiers and a flexible input multiplexer. ADS1299 is fully programmable through the SPI communication interface. In the MaxSens design, it can be configured to use up to eight differential or single ended referenced inputs. An integrated driven-right leg amplifier is used as the active neutral electrode. The EMG unit is galvanically isolated from the electrical stimulation unit which allows it to be used safely during electrical stimulation. The gains can be set to 1, 2, 4, 8, 12 or 24 and a sampling frequency to 250, 500, 1000 or 2000 Hz.

Power supply

The input power source is a single cell Li-Poly battery with the capacity of 560 mAh. The power supply unit generates + 3.3 V for the microcontroller, + 50 V for supplying the current controlled H-bridge, galvanically isolated power supply for the EMG unit and battery charging through the USB port.

Stimulation and recording electrodes

The custom-designed EMG and stimulation electrodes are shown in Fig. 3a. The EMG electrode comprises eight pairs of circular pads for bipolar EMG recording and three circular reference pads. Both electrodes were produced by screen-printing biomedical conductive Ag/AgCl and dielectric inks over 125 µm thick PET film. All electrode pads were covered with conductive hydrogel (AG725, Axelgaard, Denmark) to improve the electrode–skin contact.

Fig. 3
figure3

The image a shows the dimensions of the MaxSens system and of the recording (top) and stimulation (bottom) electrode, where b shows the placement of EMG electrode (proximal) and stimulation electrode (distal). In the actual application, an elastic band was applied over the electrodes to secure the electrodes and improve contact with the skin, while the MaxSens system was placed on the top of the band secured by a Velcro attached to the bottom of the stimulator

System operation

Firmware

The firmware for the embedded system was written in C programming language using Microchip Technology’s Atmel Studio 7 integrated development platform. A communication, stimulation and EMG recording task are executed independently and communicate with each other using the microcontroller’s hardware resources and interrupt driven firmware organization.

The communication task is responsible for the bidirectional Bluetooth and USB communication and receives textual commands from a host PC specifying stimulation and recording parameters. The received commands are then interpreted and responded to through bi-directional data exchange between the communication task and the two other tasks (namely, stimulation and EMG recording). When the commands for setting up the stimulation parameters are received, the parameters are checked, and only if they are internally consistent with the other parameters (see Section Electrical stimulation), the command is acknowledged (responding with “OK”), otherwise, it is rejected (responding with “ERR”).

The stimulation task is responsible for stimulus generation. Based on the stimulation parameters received from the communication task, the stimulation task controls the stimulus generation units in precisely defined time intervals, and sends the stimulation parameters to the EMG recording task to control the blanking algorithm, which is explained in the next section.

The EMG recording task adjusts the amplifier parameters, sampling frequency, gain for each channel and input electrode configuration (differential or single-ended reference) according to the parameters received from the communication task. The EMG amplifier blanking period and the onset of stimulation are received from the stimulation task in order to execute the stimulation blanking algorithm. Finally, this task transfers the collected EMG samples to the communication task so that they can be transmitted to the host PC. Importantly, the control of stimulation and recording consumes negligible MCU time (max 4% in the “worst” case) as these functions are performed in the background by dedicated peripheral modules. For instance, to generate an edge of a stimulation pulse, the MCU needs only to configure the compare module in response to an interrupt. Similarly, once the EMG signals are sampled by the EMG unit (ADS1299 IC), the MCU receives an interrupt to collect the data via the SPI interface. These interrupt routines are of highest priority, they are almost instantaneous, and hence most of the MCU time is in fact available for other tasks (e.g., parsing protocol messages).

Dynamic blanking

In order to suppress stimulation artifacts in EMG recordings, a novel blanking technique was implemented. Unlike commonly used methods based on hardware solutions in which the EMG amplifier inputs are shorted, the proposed technique is based on a firmware solution. The EMG amplifier is continuously active, even during stimulation pulse generation, and the number of EMG samples which need to be blanked are simply replaced with the last sample acquired before the blanking period (sample-and-hold technique). This approach is possible because of the EMG amplifier characteristics: DC amplifier topology, relatively low gain (up to 24) and a very short recovery time after amplifier saturation. Additionally, the implemented overvoltage protection circuits provide adequate amplifier inputs protection against overvoltage caused by the stimulation.

This blanking technique requires precise synchronization between the EMG unit and the electrical stimulation unit by the MCU. The blanking starts at the same time as the leading edge of the stimulation pulse and the period in which the EMG amplifier has to be blanked is computed before pulse generation based on the electric charge delivered by the pulse (amplitude × duration). The period is expressed as the number of blanked samples (NBS) of the EMG signal. The relation between NBS and the parameters of the stimulation pulse was experimentally derived. The tests were conducted using two adjacently placed EMG and stimulation channels since this positioning leads to strongest artefacts. The stimulation frequency was set to 50 Hz, the EMG amplifier gain to 24 (maximal value) and the sampling frequency to 2000 Hz. The pulse width was increased from 200 to 1000 µs in steps of 50 µs and, for each pulse width, an amplitude sweep from 1 to 10 mA in steps of 1 mA was performed. For each combination of pulse width and amplitude \(({T}_{pulse}, {I}_{pulse})\), the duration of the generated stimulation artifact was measured as the number of corrupted EMG samples. The experimental data relating the pulse width and amplitude to NBS were fitted using a first order linear polynomial surface model: \(\mathrm{z}\left(\mathrm{x},\mathrm{y}\right)= {\mathrm{p}}_{00}+{\mathrm{p}}_{10}\mathrm{x}+{\mathrm{p}}_{01}\mathrm{y}+{\mathrm{p}}_{11}\mathrm{xy}\), where (x,y) are pairs Tpulse and Ipulse, and z(x,y) is NBS (Matlab function “fit” with the model type “poly11”). The coefficients were estimated from 95% confidence intervals. The polynomial model was then expressed in the product from, shown in Eq. (1), as that form was more convenient for embedded implementation. The resulting expression is:

$$NBS = \left( {0.25 \cdot I_{{pulse}} + 7.5{\text{ mA}}} \right) \cdot \left( {0.6 \cdot T_{{pulse}} + 500\,\upmu {\text{s}}} \right) \cdot \frac{{F_{{EMG}} }}{{10^{6} \,{\text{mA}}}}$$
(1)

where \({I}_{pulse}\) represents the amplitude of the stimulation pulse in mA, \({T}_{pulse}\) is the stimulation pulse duration in µs, \({F}_{EMG}\) is the sampling frequency of the EMG amplifier, and \(NBS\) represents the number of EMG samples to be blanked (i.e., replaced by the value of the last sample recorded before stimulus generation). Therefore, increasing the current amplitude and pulse width will prolong the blanking period. Higher stimulation frequency will increase the number of blanked samples. The Eq. (1) was tested on all the recording/stimulation electrode pairs, and the validity of the equation was thereby confirmed.

However, the Eq. (1) is determined using a single active channel. When multiple stimulation channels are active, multiple pulses will be delivered within a single stimulation period separated by inter-pulse intervals (asynchronous protocol, as described above). In this case, the blanking will start with the first pulse and include the biphasic pulse width and inter-pulse interval of all pulses apart from the last pulse in the sequence (blue area in Fig. 2). The blanking period related to the last pulse will be computed according to Eq. (1) (green area in Fig. 2). The NBS in case of multiple active channels is therefore given by:

$$NBS = \left[ {\left( {0.25 \cdot I_{{pulse}}^{N} + 7.5{\text{ mA}}} \right) \cdot \left( {0.6 \cdot T_{{pulse}}^{N} + 500\,\upmu {\text{s}}} \right) + \sum\limits_{{i = 1}}^{{N - 1}} {\left( {2 \cdot T_{{pulse}}^{i} + T_{{TBP}}^{i} } \right)} } \right] \cdot \frac{{F_{{EMG}} }}{{10^{6} \,{\text{mA}}}}$$
(2)

where \({I}_{pulse}^{N}\) and \({T}_{pulse}^{N}\) represent the amplitude and duration of the last stimulation pulse in mA and µs, respectively, \({T}_{pulse}^{i}\) represents the duration of a single phase of the ith pulse in µs, while \({T}_{TBP}^{i}\) represents the inter-pulse interval time between the ith and (i + 1)th pulse in µs. The parameter \({F}_{EMG}\) represents the sampling frequency of the EMG amplifier, and \(NBS\) is the number of blanked EMG samples. Thus, increasing the number of active stimulation channels and/or inter-pulse interval will increase the NBS.

Experimental assessment

Ten subjects (eight males and two females, right-handed with a mean age of 27.1 ± 2.4 years) were recruited to test the performance of simultaneous recording and stimulation. All subjects participated voluntarily and had no musculoskeletal or skin disorders. Before participating in experiments, the subjects read and signed a consent form and were informed about the study’s methods and objectives. The study was conducted in accordance to the Declaration of Helsinki and approved by the Ethical committee of the Nordjylland Region, Denmark (approval number: N 20 150 075).

Experimental setup

Figure 3b shows the electrode placement and their connection to the MaxSens system. The electrodes are flexible and thus the end of the electrode is bent and plugged into flat ZIF connectors. The stimulator and the electrodes were placed on the dominant forearm. The EMG electrode was positioned circumferentially around the forearm approximately 3 cm distal to the elbow crease. The reference pads were placed on the condyle of the elbow joint and one distally on the upper arm. The forearm was not shaved nor prepared in any other manner for the EMG recording to best mimic the envisioned application. The stimulation and recording electrodes were placed circumferentially to make the setup more compact. Similar array configurations have been used before [27, 44, 45]. Importantly, MaxSens system is not specific to the electrode type used in the present study and it could be connected to electrodes of different shapes, sizes and pad configurations (e.g., matrix versus array). Sports tape and elastic sports bands were wrapped around the electrodes to ensure good electrode–skin contact and to have a stable surface on which to attach the Velcro-underside of the stimulator.

During the experiment, the subject was seated in a comfortable chair with the dominant forearm held vertically down along the side of the body. A 14″ monitor laptop PC was placed on a table approximately 50 cm from the subject, and was used to provide visual feedback in the experiment. The PC controlled the stimulation and received the recorded EMG signals via a USB connection to the MaxSens system. The online control loop and experimental tasks were programmed in Matlab 2019b (MathWorks, Massachusetts, USA).

Signal processing and myoelectric control

The signal processing pipeline was implemented on the host PC to control a cursor (virtual prosthesis) along two degrees of freedom, as in [28]. The MaxSens system acquired the EMG data and streamed them to the host PC, which translated myoelectric control signals into  cursor movement and sent stimulation commands to MaxSens system providing tactile feedback to the subject.

The hand opening and closing, wrist pronation and supination and rest, as well as contraction intensity, were estimated from the EMG and mapped to up, down, left and right movement of the cursor, while the movement velocity was proportional to the contraction intensity. This control scheme was selected because it can be used to control a hand prosthesis with an active wrist (hence two DoFs). Figure 4 shows a planar area within which the cursor was moved and the mapping between the movement classes and cursor directions. To further approximate the practical use of a prosthesis, the maximum velocity of the cursor was set to the maximum rotation velocity of Michelangelo Hand (Otto Bock, Duderstadt, Germany), i.e., the full rotation is performed in approx. 3 s.

Fig. 4
figure4

Illustration of the grid used in the planar target reaching task where the horizontal and vertical axes corresponded to wrist rotation and hand aperture, respectively. The black cursor is located at the initial position, which indicates a neutral hand configuration, i.e. neutral wrist and hand fully opened. The cursor moved continuously in response to subject’s myoelectric commands while the grid cells indicated the ranges in each DoF that were mapped to the levels of discrete feedback (see text for details)

The EMG signals were acquired with a sample rate of 2 kHz and transmitted to the host PC using a USB connection. Channel 5 was excluded from the data acquisition as it recorded no signal. The EMG signals were band-pass filtered between 15 and 350 Hz using a 2nd order Butterworth filter. Notch filters were applied to remove power line noise at 50 Hz and its two harmonics. Finally, a 10-sample Hampel filter was applied to remove outlier values, which were present occasionally during stimulation. The commonly used time domain features, namely, mean absolute value, zero crossings, slope sign changes and waveform length, were extracted [46]. The features were extracted in windows of 300 ms with 100 ms overlap, in order to yield a high classification accuracy with a fast update time [47]. The linear discriminant analysis (LDA) was used for classification while multiple linear regression models (one per class) were applied to estimate muscle activation level, providing thereby sequential and proportional control along the two DoFs [34, 48,49,50,51].

During online control with the stimulation active, the EMG signals contained blanked segments. All the blanked segments were identified and removed, as recommended in [37], and hence only the “useful” data were considered in each window, thereby decreasing the effective window length. As explained in [37], this approach maintains classification accuracy while preserving the update time.

Electrotactile stimulation

To address the specific aim of the present study, the electrotactile feedback was designed to maximally stress the myoelectric pipeline, while still transmitting to the subject purposeful and intuitive information that is useful for closed-loop control. The feedback scheme therefore included both amplitude and frequency modulation since increasing these two parameters produces stronger and more frequent stimulation artefacts and therefore longer blanking (cf. Eq. (1)). In addition, both parameters were modulated through the full range that is relevant for practical applications. As described below, the amplitude was changed from sensation to discomfort threshold to produce clearly perceivable sensations that were not uncomfortable. The maximum frequency was set to 50 Hz since this value evokes a fused sensation, and it becomes more difficult to discriminate the frequencies that are higher. Finally, spatial modulation, in which the information is coded by activating different electrodes of the array, is also included in the scheme so that the stimulation is delivered in the vicinity of all the recording electrodes.

The designed feedback scheme provided information regarding hand aperture and wrist rotation (see Fig. 5). While the control was continuous, as explained in the previous section, the feedback transmitted discrete information; the approach adopted in several other studies [27, 28, 52,53,54]. Each DoF was divided into five intervals. In Fig. 4, this is indicated visually by dividing the area into a grid, where each cell corresponds to a combination of intervals in the two DoFs. One cell indicated the neutral position (neutral wrist and hand fully opened), eight cells corresponded to the movement along a single DoF (four for hand aperture and four for wrist rotation), while sixteen remaining cells indicated a combination of the two DoFs (both DoFs outside the neutral position). No stimulation was provided when the cursor was located in the cell indicating neutral position. Information on the wrist rotation was provided by sequentially activating the groups of 4 pads to mimic the direction of the rotation movement. Therefore, the subject perceived a sensation over a relatively large area (4 pads simultaneously active) that would rotate medially during wrist pronation and laterally during wrist supination. The assumption was that the spatially congruent mapping would facilitate the interpretation of feedback.

Fig. 5
figure5

Illustration of the developed feedback-coding scheme. The blue circles represent electrode pads, which are connected in groups activated together (black lines), and the orange toned rings indicate the levels of amplitude and frequency. The level of wrist rotation was communicated by activation of four neighboring pads. Supination and pronation were conveyed by activating groups of pads on the lateral and medial side of the forearm, respectively. A decrease in hand aperture was communicated by a simultaneous increase of the amplitude and frequency of the currently active group of pads. No stimulation was provided at neutral position, i.e. neutral wrist and hand fully opened

The decrease in the hand aperture was conveyed as an increase in current amplitude and stimulation frequency in the four active pads, thus, mimicking a more tightly closed hand. The frequency was modulated from 10 Hz in the first interval (indicating hand fully open) to 50 Hz in the fifth interval (closed hand) using steps of 10 Hz. The stimulation amplitude was changed between sensation threshold at the first interval to just below the discomfort threshold at the fifth interval. The amplitude values for the intermediate intervals were equidistantly separated in the range between the first and fifth interval. In this way, the amplitude values were maximally separated within the dynamic range to facilitate discrimination of the feedback levels by the subject. Although MaxSens system allows fine modulation of the pulse amplitude and frequency, 5 discrete levels were used in the present study to allow the subject to interpret the feedback easily, i.e., he/she needs to memorize “only” five distinct sensations, which can be achieved after a brief training [55]. The pulse width was constant and set to 300 μs. The pulse width was selected based on our previous experience to allow fine modulation in the intensity of elicited sensations while still ensuring that the discomfort threshold can be reached in all subjects when changing the pulse amplitude. Therefore, the subject perceived the sensation that would become stronger and faster as the virtual hand was closing. The assumption was that simultaneous change in both parameters (double cuing) would assist the subjects in discriminating the levels and interpreting the feedback.

Experimental protocol

The experimental protocol for the assessment of closed-loop myoelectric control was similar to that described in [28] and it will be summarized hereafter. The system was placed on the subjects as explained in Section Experimental setup. The psychometric parameters (sensation and discomfort thresholds) were determined for each channel. The subjects then performed a maximum long-term voluntary contraction (MLVC), where they have been asked to contract their muscle strongly but at the level that can be maintained for 15 s without getting fatigued [56]. To collect the EMG data for the training of the myoelectric classifier, the subject was asked to perform each movement while tracking a trapezoidal trajectory with a cursor moving horizontally with time and vertically relative to the subject’s contraction intensity [57]. The contraction level was estimated by averaging the mean absolute value of the windowed EMG signal over all channels. The trajectory consisted of a 3-s incline, a 5-s plateau and a 3-s decline. Three recordings of trajectory tracking were acquired per movement class, with plateaus at 40%, 50% and 70% of the MLVC. Lastly, a 15-s recording of the rest class was acquired. Pilot tests showed a slightly higher EMG baseline when the stimulation was activated compared to no stimulation. Therefore, the data for the rest class was recorded while eight stimulation pads were activated (two active pads interleaved by two inactive pads etc.) at just above the sensation threshold.

After training the classifier, the subjects performed the experimental task. First, the subjects practiced the myoelectric control, after which they learned to understand the feedback scheme. Lastly, the subjects practiced the myoelectric control combined with tactile feedback (closed-loop control). To teach the subjects how to interpret the feedback coding, they went through familiarization and reinforced learning. In the familiarization phase, the visual and tactile feedback indicating movement along single DoFs was presented to the subject. Hence, he/she could associate the visual cue to the tactile sensation. In the reinforced learning phase, the subject was blinded and presented with the feedback for each combination of DoF intervals (all grid cells), and then asked to guess the indicated intervals. The subject was informed about the correct answer following a wrong estimate. The reinforced learning was finished after all grid cells were presented twice (two times 24 trials). The feedback training lasted approximately 15 min per subject.

After this brief training, the subjects performed target-reaching tests in three conditions [45, 58,59,60,61]. The first condition was the benchmark assessment, where only visual feedback was provided. The grid shown in Fig. 4 was displayed on the screen and the target cell was indicated in red. As explained before, the cursor moved continuously while the visual feedback indicated the cell in which the cursor was located (discrete feedback). If the subject reached the target cell within 30 s and remained in the target for 2 s (dwell time), the task was accomplished successfully, otherwise it was counted as a failure. In the second condition (“combined feedback”), the electrotactile feedback was delivered together with visual feedback to evaluate whether the stimulation affected the control performance. In this condition, the assumption was that the subjects would primarily rely on vision, since tactile and visual feedback provided the same information, which was easier to interpret by looking at the screen compared to decoding the tactile patterns. Hence, the electrical stimulation in this case served mainly as an interference to the EMG recording. In the last condition, only electrotactile feedback was provided, to assess whether the feedback was clear and intuitive enough for the subject to interpret it and use it for control. This assessment was performed twice using a different sequence of target cells in each run. The benchmark test was conducted after training the myoelectric control. The subject performed the other two tests after practicing the feedback scheme. In each test, all grid cells were used as targets except for the cell corresponding to the neutral position (neutral wrist and hand fully opened), hence 24 targets per test in total. The total experiment duration was between 2 and 3 h.

Data analysis

The outcome measures were completion rate, time to reach a target, path efficiency and the number of overshoots. These are common measures to evaluate the quality of control in target-reaching tasks [45, 58], and they were also used in our previous study [28]. The completion rate was the percentage of successful trials. The time to reach a target was the time in seconds from when the trial started until the target was reached including the dwell time. The path efficiency was computed by dividing the lengths of the optimal and generated path. For single-DoF targets, the optimal path was a straight line from the starting point to the border of the target, and for combined-DoF targets, it was the shortest path to the nearest corner of the target from the starting point using horizontal and vertical lines only. The number of overshoots was counted as the number of times the cursor entered a target but then moved out before the dwell time expired. This measure was used to assess the stability of control (e.g., potential drift in control due to interfering electrical stimulation) and it is reported as the mean number of overshoots per trial.

The outcome measures were calculated for each trial, after which the results were averaged within subjects for each evaluation test. As there was no significant difference in performance between the two tests with tactile feedback only, the respective within-subject outcome measures were computed as the average of the results achieved in the two tests. Only successful trials were considered relevant for the analysis, and therefore, unsuccessful trials were left out in the computation of all outcome measures besides the completion rate. Non-parametric statistics were employed as the data showed not to be normally distributed following a Kolmogorov–Smirnov test. A Friedman test was applied to assess significant differences between the test conditions, and a Tukey–Kramer test was used for post-hoc pairwise comparison. A significance level was set to p < 0.05. The results in the text are reported in the form of median/interquartile range (IQR).

Results

Effect of dynamic blanking algorithm

Figure 6 shows an example of the data collected using the developed system while the subject performed hand closing. Before plotting, the signals were filtered as explained in Section Signal processing and myoelectric control, and they are reported in arbitrary units. While the subject performed the movement, the electrotactile stimulation was delivered at amplitude level 3, pulse width of 300 µs and frequency of 30 Hz with 8 channels active as in the baseline EMG acquisition (see Section Experimental protocol). The figure indicates that the system can record EMG of good quality, with contraction bursts that were clearly visible and different across channels. However, the electrotactile stimulation substantially corrupted the recorded EMG signal. The induced artifacts were large and with variable amplitude, but they disappeared when the blanking was activated. Therefore, the dynamic artefact-blanking successfully eliminated the stimulation interference. Nevertheless, the stimulation did introduce some low-amplitude artifacts in the recording, which resulted in a higher amplitude of the baseline EMG. Specifically, during hand resting without stimulation, the baseline EMG was approximately four times lower than with the stimulation (6.4% ± 5% normalized to MLVC amplitude of each subject vs. 1.7% ± 0.9% without stimulation).

Fig. 6
figure6

EMG recorded when the subject performed hand closing with concurrent electrotactile stimulation. The signals are reported in arbitrary units. In the left recording, the blanking was activated, while it was inactive in the right recording. The recording without blanking was corrupted with pronunced artifacts that are larger in magnitude than the generated EMG

Figure 7 illustrates how frequency of stimulation affects the occurrence of blanking periods within a single window of EMG data (300 ms/600 samples). In the first recording, (Fig. 7a), parameters were set to level 2 amplitude (~ 2 mA), 20 Hz frequency, 300 µs pulse width and four channels were activated with a 100 µs inter-pulse interval. This resulted in approximately 14 blanked samples per stimulation pulse, which agrees with Eq. 2. In a single window, 7 segments of blanked data (constant value due to sample-and-hold) occurred leading to a loss of 16% of EMG data. In the second and third recordings (Fig. 7b and c), the frequency was increased to 50 Hz and 80 Hz, respectively, which resulted in 18 and 29 blanked segments within a single window corresponding to a loss of 39% and 64% of EMG data, respectively. Calculating the number of blanked samples for the feedback coding scheme used in the present study (Fig. 5), the data loss was in the range of ~ 7 to ~ 40% per window when moving from hand fully open to hand fully closed.

Fig. 7
figure7

300 ms windows of EMG recorded while the stimulation was delivered at three different frequencies: a shows a recording made using 20 Hz frequency, level 2 amplitude, 300 µs pulse width and four active channels, while in b the frequency was changed to 50 Hz and in c to 80 Hz. The red lines indicate blanked segments of the signal. In these segments, the EMG value was constant since it was “frozen” by the sample-and-hold (see Section Dynamic blanking)

Experimental task

Example trajectories from the three feedback conditions can be seen in Fig. 8. The traces were smooth for all feedback conditions with only few path corrections indicating that the myoelectric control was not affected by the electrical stimulation.

Fig. 8
figure8

Examples of trajectories generated while using visual (brown traces), combined (blue traces) and tactile feedback (yellow traces). The black circle indicates the starting point, the green circles indicate end points when the target was reached and the red squares are the targets

A summary of the results characterizing the quality of closed-loop myoelectric control in the three feedback conditions are shown in Fig. 9. There was no significant difference between the visual and combined feedback condition in any outcome measure and in both conditions the subjects achieved high completion rates (median/IQR of 100/4% for visual and 100/4% for combined). Seven subjects successfully reached all the targets in the visual and combined conditions, respectively.

Fig. 9
figure9

Box plots of the outcome measures obtained in the visual, combined and tactile feedback conditions: a completion rate, b time to reach a target, c path efficiency and d the number of overshoots. The red lines, blue boxes and black whiskers represent medians, interquartile ranges and maximum and minimum values, respectively, while the red crosses are outliers. The asterisks indicate significant differences (* is p < 0.05, ** is p < 0.01 and *** is p < 0.001)

As expected, the performance in the tactile feedback condition was worse compared to that achieved using visual and combined feedback. The decrease in performance was statistically significant in completion rate (78/25%) and time to reach a target (13/4 s). The median path efficiency (38/8%) was lower and the number of overshoots (0.5/0.4 overshoots) higher, but the difference in these two cases was not statistically significant. The completion rate of the target-reaching task with tactile feedback and the success rate in recognizing the tactile codes in the reinforced learning (74/13%) were not significantly different (Wilcoxon signed rank test).

Discussion

In this study, we presented and evaluated a novel compact electronic interface with an embedded dynamic artefact blanking algorithm to allow simultaneous EMG recording and electrotactile stimulation on the ipsilateral forearm. To the best of our knowledge, this is the first embedded system in the context of online closed-loop control of hand prostheses using electrotactile stimulation, where multichannel recording and stimulation electrodes can be placed in close proximity without signal distortion and therefore potentially integrated in a prosthetic socket. The closed-loop myoelectric control and the effectiveness of the blanking algorithm was evaluated in a target-reaching task, in which both visual and electrotactile feedback were provided, and the performance was compared to that achieved in the benchmark condition with only visual feedback. Furthermore, the degree to which the tactile feedback could be exploited for control was evaluated in a condition where only electrotactile stimulation was delivered to the subject.

Importantly, the performance in visual and combined feedback conditions was similar in all outcome measures. A high completion rate was expected with the visual feedback, and the fact that it did not change with the addition of electrotactile stimulation demonstrates that myoelectric control was not affected by the stimulation. This does not automatically mean that the control was not disturbed, as the subjects could have used visual feedback to compensate for the disturbance and maintain similar success rate. However, the fact that the time to reach a target and path efficiency did not change significantly implies that the blanking algorithm was indeed efficient and that the loss of information due to blanking did not disturb the classification performance (Figs. 8 and 9).

Nevertheless, the stimulation did produce an increased baseline EMG amplitude. However, the baseline shift was still low, and did not affect the control performance as the rest class could be activated reliably by the subjects. The change in the baseline was easily accommodated by performing an additional recording with the stimulation active, as described in Section Experimental protocol. Concentric electrode configurations reduce the current spread and could, therefore, be a method to suppress this change in the baseline [25].

Another constrain introduced by the blanking algorithm was the loss of information (blanked samples). As demonstrated in Section Effect of dynamic blanking algorithm, the loss increases substantially with frequency, and effectively no data would remain (all samples blanked) for the frequencies > 132 Hz, when using the same parameter settings as for the recording in Fig. 7. Therefore, despite the stimulator can generate the pulses at up to 400 Hz, the effective maximum frequency is far lower when the stimulation is combined with simultaneous recording of EMG. This is however not a critical limitation. The present results demonstrated that the frequency can be safely increased to 50 Hz without affecting the performance, and this is still within a preferred range of frequencies for electrotactile stimulation [62].

The conditions with visual and combined feedback are not enough to prove the applicability of the system, as the electrotactile stimulation could have been too weak to produce a significant interference. Therefore, the condition with the tactile feedback only was included in the protocol to demonstrate that the feedback was meaningful for the subjects. And indeed, the target-reaching test with only tactile feedback showed a good completion rate (~ 80%) and no significant difference compared to the visual and combined conditions for the path efficiency and the number of overshoots. These results confirm that the feedback could be successfully exploited by the subjects to efficiently navigate and maintain the target DoF configuration. Significantly lower completion rate and longer time to achieve a target with respect to visual feedback was an expected result. Similarly, it was not expected that adding stimulation (tactile feedback) would improve performance when visual feedback was also available. Indeed, the dominance of visual over tactile information is a well-known challenge in the design of effective somatosensory feedback interfaces in upper limb prostheses [8, 63]. However, the aim of the present study was to validate the system by demonstrating that it can record EMG while eliciting clear and meaningful tactile sensations. Investigating the potential benefits of such feedback will be addressed in future work. In general, the tactile feedback is likely to be beneficial when the vision is impaired or unavailable, as in the tactile only condition of the present study, and/or when the tactile channel conveys variables that are not directly observable by vision (e.g., EMG biofeedback [64]). Importantly, the present study showed that MaxSens device is effective and versatile enough to implement such closed-loop scenarios. For instance, in the present experiment, the system conveyed proprioceptive information, but other variables could be similarly mapped to the electrode array (e.g., contact [65], grasping force [11], myoelectric signal [54]), leading to a clinically useful feedback.

A specific advantage of electrotactile stimulation is the ability to change the parameters independently leading to a rich repertoire of coding schemes to transmit feedback information. The feedback encoding used in the present study modulated the current amplitude, stimulation frequency and spatial activation of electrode pads to communicate two feedback variables (aperture and rotation). As explained before, the aim was to stress the myoelectric system with different types of disturbances and the results demonstrated that the myoelectric control was robust. This implies that the presented system can be used to implement all relevant encoding approaches (i.e., parameter and spatial modulation) in different combinations or individually, without affecting the EMG recording and myoelectric control. This is an important conclusion for the applicability of the developed system to different closed-loop myoelectric scenarios.

For instance, the system could be used to implement two feedback mappings presented in [28], which were based on amplitude and spatial modulation, respectively, while the frequency was constant. These feedback schemes were evaluated using the same test as in the present study, resulting in the median completion rates that were significantly better than in the current experiment (94/10% and 94/2% versus 78/25%). However, those coding schemes where exclusively designed to be intuitive, while the feedback in the present study was constructed to challenge the myoelectric control. Contrary to [28], where the stimulation and recording were performed by two different systems placed on separate arms, the present assessment used a single unit. Therefore, this shows that proprioceptive information can be communicated intuitively via electrotactile stimulation, while activating the muscles and recording myoelectric signals from the same arm.

Some subjects reported a change in sensation quality and localization when performing movements, in particular rotation, although the stimulation parameters were constant. This change in sensation would occasionally misguide the subjects during the evaluation test with no visual feedback. However, the overall impact on performance was likely small since there was no significant difference between reinforced learning (stimulation without muscle activation) and target reaching test. A possible reason for the change in sensation could be that different skin afferents were activated by the stimulation when rotating the wrist or that the electrode–skin adhesion was altered [66]. This could be also an additional contributing factor for higher performance in [28], where the sensation was not disturbed by muscle activation and forearm movements (due to the contralateral placement of recording and stimulation unit). Although this effect is likely to be smaller for amputees, it should be taken into consideration when future feedback schemes are designed for simultaneous EMG recording and electrotactile stimulation on the ipsilateral forearm.

The assessment of the system was conducted in able-bodied subjects which is appropriate considering the aim and scope of the study. The goal was to demonstrate the functioning of the system and in particular the robustness of the dynamic blanking algorithm. To achieve this, the closed-loop control was tested in same subjects across different conditions. This demonstrated that the system successfully protected the myoelectric control from the impact of stimulation. While the performance in amputees might change, for instance, due to different sensitivity and/or control capability (e.g., the pattern-classification control might be more sensitive to information loss), the performance ratio between the conditions and thereby the overall conclusion would likely remain the same. The next step in this development is the functional evaluation with the prosthesis and in this case, the participation of target users will be indeed essential. The developed electronic interface is compact and both electrodes are slim and flexible, and could be therefore easily integrated in a prosthetic socket. The added weight and interaction between the socket/splint and the forearm might influence the quality of control as well as the subjective perception of the electrotactile feedback.

Importantly, the developed system is equipped with a Bluetooth module, which allows direct connection to the state of the art prosthetic hands supporting the same interface (Michelangelo from Otto Bock and i-Limb from Össur). In the present study, the EMG data were streamed from the device to a laptop PC, where they were processed, but the embedded module have enough processing power to implement classification as well as feedback mapping within the firmware, possibly leading to a self-contained system [67].

Conclusion

This study presented a novel compact system for simultaneous recording and stimulation with a dynamic artefact-blanking algorithm and evaluated its applicability for closed-loop myoelectric control. Specially designed, flexible and slim EMG recording and electrotactile stimulation electrodes were placed adjacently to each other on the dominant forearm. The feedback communicated proprioceptive information regarding wrist rotation and hand aperture, and was designed to stress the myoelectric control while remaining intuitive to interpret. The results indicated that the electrotactile stimulation did not affect the control quality and that the developed feedback coding scheme resulted in good control. The presented system is therefore an integrated solution for an effective closed-loop control using electrotactile stimulation, which supports all relevant coding schemes and can be possibly embedded into a prosthetic socket.

Availability of data and materials

The experimental and/or analysed datasets can be made available from the leading author upon reasonable request.

References

  1. 1.

    Biddiss EA, Chau TT. Upper limb prosthesis use and abandonment: a survey of the last 25 years. Prosthet Orthot Int. 2007;31:236–57.

    PubMed  Article  PubMed Central  Google Scholar 

  2. 2.

    Cordella F, Ciancio AL, Sacchetti R, Davalli A, Cutti AG, Guglielmelli E, et al. Literature review on needs of upper limb prosthesis users. Front Neurosci. 2016;10:1–14.

    Article  Google Scholar 

  3. 3.

    Peerdeman B, Hermens H, Stramigioli S, Rietman H, Witteveen H, Huis in `t Veld R, et al. Myoelectric forearm prostheses: state of the art from a user-centered perspective. J Rehabil Res Dev. 2011;48:719–38.

    PubMed  Article  PubMed Central  Google Scholar 

  4. 4.

    Wilke MA, Niethammer C, Meyer B, Farina D, Dosen S. Psychometric characterization of incidental feedback sources during grasping with a hand prosthesis. J Neuroeng Rehabil. 2019;16:13 p.

  5. 5.

    Svensson P, Wijk U, Björkman A, Antfolk C. A review of invasive and non-invasive sensory feedback in upper limb prostheses. Expert Rev Med Devices. 2017;14:439–47.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  6. 6.

    Graczyk EL, Resnik L, Schiefer MA, Schmitt MS, Tyler DJ. Home use of a neural-connected sensory prosthesis provides the functional and psychosocial experience of having a hand again. Sci Rep. 2018;8:1–17.

    CAS  Article  Google Scholar 

  7. 7.

    Schofield JS, Shell CE, Beckler DT, Thumser ZC, Marasco PD. Long-term home-use of sensory-motor-integrated bidirectional bionic prosthetic arms promotes functional, perceptual, and cognitive changes. Front Neurosci. 2020;14:1–20.

    Article  Google Scholar 

  8. 8.

    Sensinger JW, Dosen S. A review of sensory feedback in upper-limb prostheses from the perspective of human motor control. Front Hum Neurosci. 2020;14:1–24.

    Google Scholar 

  9. 9.

    Clemente F, Valle G, Controzzi M, Strauss I, Iberite F, Stieglitz T, et al. Intraneural sensory feedback restores grip force control and motor coordination while using a prosthetic hand. J Neural Eng. 2019;16:026034 (9 p).

  10. 10.

    Valle G, Mazzoni A, Iberite F, D’Anna E, Strauss I, Granata G, et al. Biomimetic intraneural sensory feedback enhances sensation naturalness, tactile sensitivity, and manual dexterity in a bidirectional prosthesis. Neuron. 2018;100:37.e7-45.e7.

    Article  CAS  Google Scholar 

  11. 11.

    Markovic M, Schweisfurth MA, Engels LF, Bentz T, Wüstefeld D, Farina D, et al. The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis. J Neuroeng Rehabil. 2018;15(28):1–15.

    Google Scholar 

  12. 12.

    Di Pino G, Romano D, Spaccasassi C, Mioli A, D’Alonzo M, Sacchetti R, et al. Sensory- and action-oriented embodiment of neurally-interfaced robotic hand prostheses. Front Neurosci. 2020;14:1–17.

    Google Scholar 

  13. 13.

    Shehata AW, Rehani M, Jassat ZE, Hebert JS. Mechanotactile sensory feedback improves embodiment of a prosthetic hand during active use. Front Neurosci. 2020;14:1–12.

    Article  Google Scholar 

  14. 14.

    Systems V. Vincent Evolution 4 [Internet]. 2005 [cited 2019 Aug 1]. Available from: https://www.vincentsystems.de/evolution4.

  15. 15.

    Bionics M. Luke Arm [Internet]. [cited 2021 Mar 9]. Available from: https://www.mobiusbionics.com/luke-arm/.

  16. 16.

    PSYONIC. Ability Hand [Internet]. 2019 [cited 2020 May 29]. Available from: https://www.psyonic.co/abilityhand.

  17. 17.

    Antfolk C, D’Alonzo M, Rosén B, Lundborg G, Sebelius F, Cipriani C, et al. Sensory feedback in upper limb prosthetics. Expert Rev Ltd. 2013;10:45–54.

    CAS  Google Scholar 

  18. 18.

    Stephens-Fripp B, Alici G, Mutlu R. A review of non-invasive sensory feedback methods for transradial prosthetic hands. IEEE Access. 2018;6:6878–99.

    Article  Google Scholar 

  19. 19.

    Schofield JS, Evans KR, Carey JP, Hebert JS. Applications of sensory feedback in motorized upper extremity prosthesis: a review. Expert Rev Med Devices. 2014;11:499–511.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  20. 20.

    Mastinu E, Engels LF, Clemente F, Dione M, Sassu P, Aszmann O, et al. Neural feedback strategies to improve grasping coordination in neuromusculoskeletal prostheses. Sci Rep. 2020;10:1–15.

    Article  CAS  Google Scholar 

  21. 21.

    Chandrasekaran S, Nanivadekar AC, Mckernan G, Helm ER, Boninger ML, Collinger JL, et al. Sensory restoration by epidural stimulation of the lateral spinal cord in upper-limb amputees. Elife. 2020;9:1–26.

    Article  Google Scholar 

  22. 22.

    Callier T, Brantly NW, Caravelli A, Bensmaia SJ. The frequency of cortical microstimulation shapes artificial touch. Proc Natl Acad Sci USA. 2020;117:1191–200.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  23. 23.

    Engdahl SM, Christie BP, Kelly B, Davis A, Chestek CA, Gates DH. Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques. J Neuroeng Rehabil. 2015;12:1–11.

    Article  Google Scholar 

  24. 24.

    Kaczmarek KA, Webster JG, Bach-y-Rita P, Tompkins WJ. Electrotactile and vibrotactile displays for sensory substitution systems. IEEE Trans Biomed Eng. 1991;38:1–16.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  25. 25.

    Szeto AYJ, Saunders FA. Electrocutaneous stimulation for sensory communication in rehabilitation engineering. IEEE Trans Biomed Eng. 1982;BME-29:300–8.

    Article  Google Scholar 

  26. 26.

    Franceschi M, Seminara L, Dosen S, Strbac M, Valle M, Farina D. A system for electrotactile feedback using electronic skin and flexible matrix electrodes: experimental evaluation. IEEE Trans Haptics. 2017;10:162–72.

    PubMed  Article  PubMed Central  Google Scholar 

  27. 27.

    Štrbac M, Belić M, Isaković M, Kojić V, Bijelić G, Popović I, et al. Integrated and flexible multichannel interface for electrotactile stimulation. J Neural Eng. 2016;13:1–16.

    Article  Google Scholar 

  28. 28.

    Garenfeld MA, Mortensen CK, Strbac M, Dideriksen JL, Dosen S. Amplitude versus spatially modulated electrotactile feedback for myoelectric control of two degrees of freedom. J Neural Eng. 2020;17:1–15.

    Article  Google Scholar 

  29. 29.

    Mandrile F, Farina D, Pozzo M, Merletti R. Stimulation artifact in surface EMG signal: effect of the stimulation waveform, detection system, and current amplitude using hybrid stimulation technique. IEEE Trans Neural Syst Rehabil Eng. 2003;11:407–15.

    PubMed  Article  PubMed Central  Google Scholar 

  30. 30.

    Yi X, Jia J, Deng S, Shen SG, Xie Q, Wang G. A blink restoration system with contralateral EMG triggered stimulation and real-time artifact blanking. IEEE Trans Biomed Circuits Syst. 2013;7:140–8.

    PubMed  Article  Google Scholar 

  31. 31.

    Schauer T, Salbert R, Negard N-O, Raisch J. Detection and filtering of EMG for assessing voluntary muscle activity during FES. In: 9th Annual Conference of the International FES Society. 2004. p. 185–7.

  32. 32.

    Yeom H, Park H, Chang YH, Park Y, Lee KJ. Stimulus artifact suppression using the stimulation synchronous adaptive impulse correlated filter for surface EMG application. J Electr Eng Technol. 2012;7:451–8.

    Article  Google Scholar 

  33. 33.

    Keller T, Popovic MR. Real-time stimulation artifact removal in EMG signals for neuroprosthesis control applications. In: Proceedings of the 6th Annual IFESS Conference. 2001. p. 4–6.

  34. 34.

    Yang D, Gu Y, Thakor NV, Liu H. Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration. Exp Brain Res. 2019;237:291–311.

    PubMed  Article  Google Scholar 

  35. 35.

    Dosen S, Markovic M, Strbac M, Belic M, Kojic V, Bijelic G, et al. Multichannel electrotactile feedback with spatial and mixed coding for closed-loop control of grasping force in hand prostheses. IEEE Trans Neural Syst Rehabil Eng. 2016;25:183–95.

    PubMed  Article  Google Scholar 

  36. 36.

    Xu H, Zhang D, Huegel JC, Xu W, Zhu X. Effects of different tactile feedback on myoelectric closed-loop control for grasping based on electrotactile stimulation. IEEE Trans Neural Syst Rehabil Eng. 2016;24:827–36.

    PubMed  Article  Google Scholar 

  37. 37.

    Hartmann C, Došen S, Amsuess S, Farina D, Dosen S, Amsuess S, et al. Closed-loop control of myoelectric prostheses with electrotactile feedback: influence of stimulation artifact and blanking. IEEE Trans Neural Syst Rehabil Eng. 2015;23:807–16.

    PubMed  Article  Google Scholar 

  38. 38.

    Arieta AH, Yokoi H, Arai T, Yu W. Study on the effects of electrical stimulation on the pattern recognition for an EMG prosthetic application. In: Annual International Conference of the IEEE Engineering in Medical and Biology—Proceedings. 2005;7 VOLS:6919–22.

  39. 39.

    Almström C, Anani A, Herberts P, Körner L. Electrical stimulation and myoelectric control. A theoretical and applied study relevant to prosthesis sensory feedback. Med Biol Eng Comput. 1981;19:645–53.

    PubMed  Article  Google Scholar 

  40. 40.

    Dosen S, Schaeffer M, Farina D. Time-division multiplexing for myoelectric closed-loop control using electrotactile feedback. J Neuroeng Rehabil. 2014;11:1–10.

    Article  Google Scholar 

  41. 41.

    Oskoei MA, Hu H. Myoelectric control systems—a survey. Biomed Signal Process Control. 2007;2:275–94.

    Article  Google Scholar 

  42. 42.

    Malešević NM, Maneski LZP, Ilić V, Jorgovanović N, Bijelić G, Keller T, et al. A multi-pad electrode based functional electrical stimulation system for restoration of grasp. J Neuroeng Rehabil. 2012;9:1–12.

    Article  Google Scholar 

  43. 43.

    Strbac M, Isakovic M, Malesevic J, Keller T, Ilic V, Jorgovanovic N, et al. Compact electrotactile stimulation and EMG acquisition system with embedded blanking of stimulation artifacts. In: 2019 26th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2019. 2019. p. 863–6.

  44. 44.

    Witteveen HJB, Droog EA, Rietman JS, Veltink PH. Vibro- and electrotactile user feedback on hand opening for myoelectric forearm prostheses. IEEE Trans Biomed Eng. 2012;59:2219–26.

    CAS  PubMed  Article  Google Scholar 

  45. 45.

    Scheme EJ, Hudgins BS, Englehart KB. Confidence-based rejection for improved pattern recognition myoelectric control. IEEE Trans Biomed Eng. 2013;60:1563–70.

    PubMed  Article  PubMed Central  Google Scholar 

  46. 46.

    Hudgins B, Parker P, Scott RN. A new strategy for multifunction myoelectric control. IEEE Trans Biomed Eng. 1993;40:82–94.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  47. 47.

    Smith LH, Hargrove LJ, Lock BA, Kuiken TA. Determining the optimal window length for pattern recognition- based myoelectric control: balancing the competing effects of classification error and controller dela. IEEE Trans Neural Syst Rehabil Eng. 2014;19:186–92.

    Article  Google Scholar 

  48. 48.

    Scheme E, Englehart K. Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabil Res Dev. 2011;48:643–60.

    PubMed  Article  PubMed Central  Google Scholar 

  49. 49.

    Simon AM, Hargrove LJ, Lock BA, Kuiken TA. The Target Achievement Control Test: evaluating real-time myoelectric pattern recognition control of a multifunctional upper-limb prosthesis. J Rehabil Res Dev. 2011;48(6):619–27.

  50. 50.

    Parker P, Englehart K, Hudgins B. Myoelectric signal processing for control of powered limb prostheses. J Electromyogr Kinesiol. 2006;16:541–8.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  51. 51.

    Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003;50:848–54.

    PubMed  Article  PubMed Central  Google Scholar 

  52. 52.

    Isaković M, Belić M, Došen S, Štrbac M, Popović I, Keller T, et al. Electrotactile feedback improves performance and facilitates learning in the routine grasping task. Eur J Transl Myol. 2016;26:197–202.

    Article  Google Scholar 

  53. 53.

    Cipriani C, Segil JL, Clemente F, Richard RF, Edin B. Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand. Exp Brain Res. 2014;232:3421–9.

    PubMed  PubMed Central  Article  Google Scholar 

  54. 54.

    Schweisfurth MA, Markovic M, Dosen S, Teich F, Graimann B, Farina D. Electrotactile EMG feedback improves the control of prosthesis grasping force. J Neural Eng. 2016;13:15 p.

  55. 55.

    Anani AB, Ikeda K, Körner LM. Human ability to discriminate various parameters in afferent electrical nerve stimulation with particular reference to prostheses sensory feedback. Med Biol Eng Comput. 1977;15:363–73.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  56. 56.

    Amsüss S, Paredes LP, Rudigkeit N, Graimann B, Herrmann MJ, Farina D. Long term stability of surface EMG pattern classification for prosthetic control. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS. 2013;35:3622–5.

    Google Scholar 

  57. 57.

    Celadon N, Do S, Binder I, Ariano P, Farina D, Došen S, et al. Proportional estimation of finger movements from high-density surface electromyography. J Neuroeng Rehabil. 2016;13:1–19.

    Article  Google Scholar 

  58. 58.

    Scheme E, Englehart K. Validation of a selective ensemble-based classification scheme for myoelectric control using a three dimensional Fitts’ law test. Neural Syst Rehabil Eng IEEE Trans. 2013;21:616–23.

    Article  Google Scholar 

  59. 59.

    Wurth SM, Hargrove LJ. A real-time comparison between direct control, sequential pattern recognition control and simultaneous pattern recognition control using a Fitts’ law style assessment procedure. J Neuroeng Rehabil. 2014;11:1–13.

    Article  Google Scholar 

  60. 60.

    Waris A, Mendez I, Englehart K, Jensen W, Kamavuako EN. On the robustness of real-time myoelectric control investigations: a multiday Fitts’ law approach. J Neural Eng. 2019;16(2):026003.

    PubMed  Article  PubMed Central  Google Scholar 

  61. 61.

    Kamavuako EN, Scheme EJ, Englehart KB. On the usability of intramuscular EMG for prosthetic control: a Fitts’ Law approach. J Electromyogr Kinesiol. 2014;24:770–7.

    PubMed  Article  PubMed Central  Google Scholar 

  62. 62.

    Paredes LP, Dosen S, Rattay F, Graimann B, Farina D. The impact of the stimulation frequency on closed-loop control with electrotactile feedback. J Neuroeng Rehabil. 2015;12:1–16.

    Article  Google Scholar 

  63. 63.

    Adams JA, Gopher D, Lintern G. Effects of visual and proprioceptive feedback on motor learning. J Mot Behav. 1977;9:11–22.

    Google Scholar 

  64. 64.

    Dosen S, Markovic M, Somer K, Graimann B, Farina D. EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis. J Neuroeng Rehabil. 2015;12:55.

    PubMed  PubMed Central  Article  Google Scholar 

  65. 65.

    Clemente F, D’Alonzo M, Controzzi M, Edin BB, Cipriani C. Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses. IEEE Trans Neural Syst Rehabil Eng. 2016;24:1314–22.

    PubMed  Article  PubMed Central  Google Scholar 

  66. 66.

    Akhtar A, Sombeck J, Boyce B, Bretl T. Controlling sensation intensity for electrotactile stimulation in human-machine interfaces. Sci Robot. 2018;3:eaap9770.

  67. 67.

    Samuel OW, Asogbon MG, Geng Y, Al-Timemy AH, Pirbhulal S, Ji N, et al. Intelligent EMG pattern recognition control method for upper-limb multifunctional prostheses: advances, current challenges, and future prospects. IEEE Access. 2019;7:10150–65.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The work in this study was supported by the project ROBIN (8022-00243A and 8022-00226B) funded by the Independent Research Fund Denmark.

Author information

Affiliations

Authors

Contributions

NJ, VI, MS and SD conceptulized the study. MS envisoned the device and the blanking algorithm, NJ and VI designed the system. MS and MI designed the electrodes and worked on technical validation of the overall system. JLD, MAG and SD designed the experimental protocol. MAG implemented the protocol and collected the data. JLD, MAG and SD analyzed the data and drafted the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Martin A. Garenfeld or Strahinja Dosen.

Ethics declarations

Ethics approval and consent to participate

The study was conducted in accordance to the Declaration of Helsinki and approved by the Ethical committee of the Nordjylland Region, Denmark (Approval Number: N 20 150 075). Before participating in experiments, the subjects were informed about the study’s methods and objectives and they have signed a consent form.

Consent for publication

Not applicable.

Competing interests

The authors mutually declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Garenfeld, M.A., Jorgovanovic, N., Ilic, V. et al. A compact system for simultaneous stimulation and recording for closed-loop myoelectric control. J NeuroEngineering Rehabil 18, 87 (2021). https://0-doi-org.brum.beds.ac.uk/10.1186/s12984-021-00877-5

Download citation

Keywords

  • Closed-loop control
  • Myoelectric prosthesis
  • Electrotactile stimulation
  • Sensory feedback
  • Dynamic blanking