Computational neurorehabilitation: modeling plasticity and learning to predict recovery
© Reinkensmeyer et al. 2016
Received: 19 November 2015
Accepted: 13 April 2016
Published: 30 April 2016
Despite progress in using computational approaches to inform medicine and neuroscience in the last 30 years, there have been few attempts to model the mechanisms underlying sensorimotor rehabilitation. We argue that a fundamental understanding of neurologic recovery, and as a result accurate predictions at the individual level, will be facilitated by developing computational models of the salient neural processes, including plasticity and learning systems of the brain, and integrating them into a context specific to rehabilitation. Here, we therefore discuss Computational Neurorehabilitation, a newly emerging field aimed at modeling plasticity and motor learning to understand and improve movement recovery of individuals with neurologic impairment. We first explain how the emergence of robotics and wearable sensors for rehabilitation is providing data that make development and testing of such models increasingly feasible. We then review key aspects of plasticity and motor learning that such models will incorporate. We proceed by discussing how computational neurorehabilitation models relate to the current benchmark in rehabilitation modeling – regression-based, prognostic modeling. We then critically discuss the first computational neurorehabilitation models, which have primarily focused on modeling rehabilitation of the upper extremity after stroke, and show how even simple models have produced novel ideas for future investigation. Finally, we conclude with key directions for future research, anticipating that soon we will see the emergence of mechanistic models of motor recovery that are informed by clinical imaging results and driven by the actual movement content of rehabilitation therapy as well as wearable sensor-based records of daily activity.
Nature of the problem and definition of computational neurorehabilitation
At the onset, we define several terms that we will use throughout the paper, which provide a conceptual framework for computational neurorehabilitation. We will use the term “recovery” to describe improvements in movement ability over time, resulting in improvements at any of the ICF levels, regardless of how these improvements occurred. Note that this definition of recovery is similar to that used in [5, 6], but different from that proposed in , where the term is restricted to improvements resulting from restitution of normative biological structures and functions; we feel that “restitution” is indeed the more natural term for this more specific concept (we also found ourselves using the term “true recovery” in our discussions; others use the term “repair”.) Thus, for the purposes of this paper, we follow the nomenclature in [5, 6], in which recovery occurs through restitution, but also through compensation, which we define as use of biological structures and/or function different from those originally used before the injury to achieve a movement goal. Easy-to-understand examples of compensation are, after a stroke, using the less-affected side to perform tasks that one normally would have done with the more-affected side, or, reaching forward by leaning an abnormal amount with the trunk rather than using the usual amount of shoulder and elbow extension . Note that the different modes of recovery (restitution and compensation) may occur concurrently at different levels of analysis. For example, more normal movement behavior, which appears as biomechanical restitution, may result from leveraging residual neural substrate, a form of neural compensation.
We use the terms “learning” and “plasticity” as follows (and here, we are referring to motor system learning and plasticity). If people with or without a neurologic injury train at a motor task their ability to perform the task will improve through normal skill acquisition . This process of “motor learning” is dependent on plasticity, both in health and disease. In chronic stroke patients, training of appropriate tasks can therefore lead to improved function (Fig. 1b) . However, the anatomy of the damage sets a limit on how much impairments, such as degraded force production capability, can be reduced in the chronic phase. Therefore functional improvements in this phase often appear to be due to learning compensation techniques , although targeted training may allow modest reduction in specific impairments, assessed quantitatively (e.g. [8, 11–13]). The early post-stroke period is interesting in that there are a number of injury-induced changes in the potential for plasticity, including, for example, exuberant production then activity-based pruning of new synapses , that may last several months, causing spontaneous biological recovery (see below) . At least in animal models, motor training during this period appears to lead to a more rapid and generalized improvement in function through reduced impairment . A number of therapeutic interventions under investigation can be thought of as attempts to prolong or even re-open this ‘critical period’ of plasticity, for example drugs such as fluoxetine [15, 16], non-invasive brain stimulation , enriched environments  and aerobic exercise . As we will see below, so far, the interaction of this critical period with rehabilitation has not yet been well explored in computational neurorehabilitation, but it is an important target for modeling.
To make a computational analogy, recovery can be viewed as a constrained optimization problem. The amount and type of anatomical injury define the initial constraints. Unique forms of plasticity present early after injury, driven in part by experience and amenable to therapeutic interventions, act to alter the constraints, especially in early recovery. Motor learning is like the optimization itself. Motor learning that finds solutions similar to those used before the injury results in “restitution”, while motor learning that finds new solutions (which are potentially local minima) results in “compensation”. At present, it appears that intensive motor training during the early period of spontaneous biological recovery may be best suited for optimization of both surviving and new networks that results in substantial recovery of motor performance.
The idea of mathematically modeling sensorimotor recovery is not new. For example, as reviewed below, there is a rich history of research in prognostic models that take as inputs patient clinical features, baseline measurements of behavior, and/or brain imaging measurements, then predict functional outcomes at future time points using regression techniques (see reviews: [19, 20]). There are also models that have focused on altered network dynamics following injury (e.g. [21–24]), and now, the first few models that have incorporated specific aspects of rehabilitation into their dynamics (see below and related reviews [25, 26]). What is new about the computational neurorehabilitation approach is that it attempts to mathematically model the mechanisms underlying the rehabilitation process itself in order to understand the recovery of motor behavior, again via both restitution and compensation.
Specifically, we define computational neurorehabilitation models as models with three key features (Fig. 1a). Here, we describe these features in the context of sensorimotor rehabilitation, although the features can be broadened to describe other aspects of rehabilitation.
Second, computational neurorehabilitation models explicitly model computational mechanisms of activity-dependent plasticity. Here, we define “activity-dependent plasticity” as changes in the motor system that are caused by sensorimotor activity. Motor learning, which depends on activity-dependent plasticity, is often the basis for recovery through compensation (for example ) and has been the primary focus of the initial models reviewed below. Other forms of plasticity are also relevant to recovery, and these may not cause motor learning, as is clear from studies of neuronal changes early after neural injury . Computational neurorehabilitation models have internal states that have a biological or functional meaning and are dynamical in nature (e.g. Fig. 1d). This distinguishes these models from the input/output type models that have been developed for prognostic regression (as reviewed below), or models with arbitrary internal states that are not linked to neuro-recovery mechanisms.
Third, computational neurorehabilitation models produce as outputs quantitative variables that vary with time and that relate to functional outcomes. Example outputs for computational neurorehabilitation models of arm recovery after stroke are predictions of the changes in Fugl-Meyer Motor Score, or changes in detailed quantitative measures of arm function, such as arm movement kinematics or changes in statistical patterns of the daily amount of use of the arm.
How will computational models of neurorehabilitation be useful?
We foresee three main uses for computational models of neurorehabilitation. First, such models will provide a rigorous methodology for understanding mechanisms of recovery, that is, the biological entities and processes that implement recovery. As we survey in the next section, much is now known about various neurobiological processes important to effective rehabilitation. However, what is lacking is the integration of the processes, which operate at widely different spatial and temporal scales. Developing computational models of rehabilitation will force researchers to make these processes and their dynamic interactions more concrete. We thus believe that computational neurorehabilitation will become essential for providing frameworks to organize a diverse and growing body of data. Such multi-level computational approaches are already playing important roles in fields such as HIV [32, 33] and cancer treatment .
The second use of these models will be to aid in designing new clinical experiments. Currently, optimization of rehabilitation proceeds slowly in a trial-and-error fashion that is overly dependent on large and very expensive clinical trials that include multiple testing points for each participant. As in engineering design, a mechanistic, mathematical model of the system of interest will allow the effect of variations in rehabilitative parameters to be simulated, allowing a means to design potentially more effective experiments. Initial examples of this approach have already been demonstrated in the related field of motor adaptation, in which computational models of adaptation have been used to conceptualize behavioral environments that accelerate an individual’s ability to learn, e.g., , although adaptation is a somewhat limited type of motor learning to study for rehabilitation purposes. Nevertheless, we expect, by analogy, that new mathematical models of behavioral interventions relevant for rehabilitation will provide a means to conceptualize and design studies that can potentially enhance recovery. Use of models will also help guide collection of the types of data that can answer important mechanistic questions.
The third use of computational neurorehabilitation models relates to the second but extends it, and is to optimize therapy selection for individual patients, in terms of dosage, timing, scheduling, and content. How much therapy should patient X receive? At what time and according to what schedule should this therapy be provided? What movements should he or she practice with what sort of instructions and feedback? Currently, treatment modality and dose are mainly determined based on clinical opinion or historical precedent. In some cases, data from clinical trials influence these choices, but these data reflect averages from large groups of patients. Computational models of stroke recovery will enhance precision medicine and improve stratified trials by allowing better selection of patients for specific evidence-based therapies as well as optimizing the dosage of such therapies. For example, based on current knowledge about the predictive value of the shoulder-abduction-finger-extension (SAFE) model for the upper paretic limb within 72 h post stroke, a prognostic algorithm for selecting evidence based therapies was recently developed as a smartphone app . We expect in the future, that these computerized prognostic algorithms for improving task-specific treatments may be further optimized by additional information from neuroimaging  and more sensitive information from kinematic assays about quality of motor performance post stroke . Computational neurorehabilitation models will further enhance these efforts by incorporating explicit representations of plasticity and learning mechanisms, potentially improving predictive capability.
Organization of this review
Nature of the problem and definition of computational neurorehabilitation
How will computational models of neurorehabilitation be useful?
I. Model elements for computational neurorehabilitation
A. Inputs: Sensorimotor Activity
B. Innards: Modeling activity-dependent plasticity
C. Outputs: Functional outcomes and kinematics
II. The Current Modeling Benchmark: Prognostic Regression Models
A. Predicting outcome post stroke with baseline behavioral measures
B. Predicting outcome post-stroke with brain imaging measures
C. Predicting treatment effects
III. Computational neurorehabilitation models
A. Reaching the threshold for recovery in bilateral hand use
B. Recovering from weakness via reinforcement learning
C. Robot assistance, retention, and learning predicts recovery
D. Understanding interactions between function and use
E. Modeling the effect of assistance-as-needed
F. Patient-trainer dynamics as an optimization
Model elements for computational neurorehabilitation
This section reviews the key elements needed to construct a computational neurorehabilitation model, which are A) a quantitative description of the sensorimotor activity that the patient experiences; B) a computational model of the plasticity mediating recovery; and C) a quantitative description of the patient’s behavioral outcomes. To provide a specific context for the discussion, we again concentrate on strokes affecting motor control of the upper extremity, as much of the initial work in computational neurorehabilitation is being done in this area.
Inputs: sensorimotor activity
Modeling activity-dependent plasticity requires a quantitative description of activity that stimulates plasticity. Historically, sensorimotor activity during neurorehabilitation has been characterized in research studies and clinical practice primarily by the amount of time spent in assigned therapy sessions [9, 27]. It is also possible to simulate training sessions, in order to derive theoretical inputs for models, as has been done for most initial models described below. However, one reason that computational neurorehabilitation models have the potential to soon become much more elaborate and powerful is that researchers are beginning to quantify more precisely the sensorimotor activity that a patient experiences. There has been increased interest in quantitative, observational studies, and in new sensing technologies, including robotics and wearable sensors.
Observational studies of rehabilitation therapy
Recent observational studies found that although stroke patients spend approximately 47 min in occupational therapy each day in early rehabilitation, only 4–11 min of this time is focused on upper extremity rehabilitation [38, 39]. Distinguishing between total therapy time and active movement time is essential for accurately driving computational neurorehabilitation models. Another fundamental question that was only recently answered is “How many practice movements are typically made during rehabilitation therapy?” For the upper extremity after stroke, a study of 162 rehabilitation sessions in seven sites yielded an average of 32 functionally oriented movements . Notably, this number of movements per session is an order of magnitude less than the number of movements per session that has been shown to induce motor plasticity in animal models . There is evidence that upper extremity interventions can be designed to provide such a larger number of repetitions without increasing therapy duration [41–43].
Quantification of sensorimotor activity during therapy
Use of robotics and sensor-based therapies, including virtual rehabilitation [44, 45] and exergaming/serious games , has grown rapidly in both rehabilitation research and practice in the last 20 years [47–52] although the overall percentage of clinics using these new technologies is still relatively low . The primary motivation for developing these technologies is to provide a greater dose of therapy, but an important secondary benefit relevant to computational neurorehabilitation models is that these technologies can continuously measure the sensorimotor activity of the patient during therapy. For example, a robotic device that assists in therapy of the upper extremity (Fig. 1c) can measure the forces and motions that a patient experiences during training, providing insight into both the motor commands and intrinsic biological feedback that result from those commands. Such a device can also record any external, augmented sensory feedback – visual, audio, and haptic – that the patient experiences during training, since this feedback is provided by the device itself (assuming the therapist is not also providing feedback). Quantifying feedback content as well as movement itself is important because feedback powerfully modulates motor learning  and rehabilitation .
Examples of the type of data available from robotic and sensor-based therapy devices include the number of movements made and the trajectories achieved while making these movements. Other key variables relate to kinetics, such as the forces applied by the robot to the patient [56, 57] or by the patient to the robot , or amount of positive and negative work done on the patient during therapy with the device [59, 60]. Isolated sensors can also quantify the physical interaction forces and motions that therapists apply during hands-on therapy . Such biomechanical measurements can be combined with measures of Electromyography (EMG) to generate estimates of muscle activity during training, and, increasingly, brain imaging techniques, including Electroencephalography (EEG) , Near-Infrared Spectroscopy (NIRS) , and functional Magnetic Resonance Imaging (fMRI) , to provide insight into brain activity during training.
Quantification of sensorimotor activity during daily activities
In rehabilitation research there has been an increasing recognition that the sensorimotor activity experienced during therapy is only one part of the total sensorimotor activity that drives recovery, or, put another way, that daily use of the arm likely also plays a major role in aiding recovery . Again, new technologies, in this case wearable sensors, are now making it possible to quantify this daily activity beyond how it was done in the past [66, 67], i.e. through patient self-report scales such as the Motor Activity Log . The primary approach being used so far for the upper extremity is wrist accelerometry, in which a three-axis accelerometer is embedded in a wristband [69–72].
Wrist accelerometry is typically used to detect the amount of time spent moving the arm using a thresholding approach . If sensors are worn on both arms, the amount of bimanual activity can be quantified, and the activity of the stroke-impaired arm can be compared to that of the less affected arm . Indeed most human motor activity seems to be bimanual in nature [74, 75], which has implications for how computational neurorehabilitation models should be structured. New wearable sensing approaches are making it possible to non-obtrusively quantify finger and hand activity as well as gross arm movement during daily life [76, 77].
Innards: modeling activity-dependent plasticity
Given a quantitative description of sensorimotor activity during stroke recovery, a computational neurorehabilitation model uses this description to drive a mathematical model of activity-dependent plasticity mechanisms. Here, we briefly overview two types of activity-dependent plasticity that will play a key role in computational neurorehabilitation models – one related to spontaneous biological recovery, and one related to motor learning. For reviews see [78–80]. Note that for ease of presentation we speak of plasticity and learning rules as if they were independent form the model structure, but for most models they will not be. The model will need to consider how the necessary anatomical and functional structures support learning and plasticity, regardless of the abstraction level of the model.
Spontaneous biological recovery and activity-dependent plasticity
Many initial performance changes after stroke are attributed to “spontaneous biological recovery”, a term that implies that this recovery is automatic, although spontaneous biological recovery is almost certainly modulated by and requires behavior for maximal expression [10, 14]. Animal models indicate that spontaneous biological recovery is aided by a significantly altered tissue microenvironment triggered by the injury, in which, for example, a different profile of genes is expressed compared to during normal motor learning . Spontaneous biological recovery also involves both reduction of the ischemic penumbra and brain reorganization in areas both near the lesion and farther away . Spontaneous biological recovery is maximally expressed in the first several weeks post-stroke, and tapers off over months [10, 14, 20, 82]. Brain reorganization processes underlying this spontaneous recovery are thought to be driven by homeostatic mechanisms, Hebbian-like processes driven by long-term potentiation (LTP) [83, 84], as well as spine, dendritic and axonal forms of structural plasticity.
Soon after stroke, abnormal cortical patterns of excitation and inhibition occur both near [84–87]) and far from the lesion . Homeostatic plasticity, which is ubiquitous in the brain, acts to maintain desired firing rates and patterns . After a lesion, because of the loss of interneuronal connections, the activities of neurons neighboring the lesions, or neurons previously connected to neurons within the lesion, are affected. Homeostatic plasticity may be crucial for network recovery, as measured by re-establishment of lesion-affected inputs . In addition, sensorimotor activity might modulate this homeostatic plasticity, which is of importance for computational neurorehabilitation models, as it is one example of how sensorimotor activity appears to modulate spontaneous recovery [6, 31, 89, 90].
LTP, LTD and neural structural plasticity such as dendritic and axonal sprouting, are also modulated by sensorimotor activity, and also change as a function of time. Following stroke, some features of brain function revert to those seen at an early stage of development, with the subsequent process of “recovery recapitulating ontogeny” , but there is also a distinct, age-related pattern of gene expression, a “recovery transcriptome” . Genetic changes in the perilesional area allow for a window of increased plasticity that makes it easier for the perilesional neurons to modify existing connections and form new ones in response to sensorimotor activity . Increased LTP may also potentially lead to maladaptive plasticity and poor cortical reorganization if existing inputs are further strengthened at the expense of the reemergence of weak afferent synapses . In summary, underlying mechanisms assumed to contribute to the non-linear time course of recovery of movement in the first 3 months after stroke presumably reflect the interaction between a period of heightened plasticity mechanisms, occurring in a limited time window, and sensorimotor activity [81, 93–95]. A practical implication is that, when new patterns of movement that are a consequence of specific combinations of muscle weakness (e.g. increased trunk flexion and abduction of the shoulder during reaching) are attempted repeatedly during this period of heightened plasticity, they may become the new ‘norm’ – hence patients get stuck in local minima. Further, use of the less-impaired arm may subvert the heightened plasticity of the stroke-affected hemisphere, preventing it from improving the paretic arm function .
There are as yet few computational models of spontaneous biological recovery, much less of the interaction between spontaneous biological recovery and sensorimotor activity. Computational models of the effects of stroke to date have primarily focused on the network effects of deleting cells or of altered connectivity. For example, one early model used a difference-of-Gaussians connectivity pattern to explain rapid changes in the size of cellular receptive fields after stroke lesions [96, 97]. Other models have studied interhemispheric effects of lesions [23, 24], and used connectome data to model brain regions as graphical network nodes, evaluating the effects of node deletion on network dynamics [98–100]. One of the first models to study the effects of network changes on movement kinematics evaluated the effect of lesion size on post-stroke reach variability using cortical cells that were tuned to preferred reach directions, but did not simulate plastic processes after lesion . A recent model studied the interaction between homeostatic plasticity and Hebbian-plasticity after stroke in the somatosensory cortex, and suggests that after a lesion, a delay preceding rehabilitation would allow a return of homeostatically-determined desired firing in cells neighboring the lesions, and thus may allow a faster network recovery in the rehabilitation training compared to no delay . It will be increasingly important to compare models that incorporate spontaneous biological recovery mechanisms to ones that do not, to determine how modeling these phenomena improves explanatory power. New analytical approaches to examine structural and functional connectivity within well-defined macroscopic brain networks, as briefly reviewed in Section II C below, will increasingly play a role, and will integrate plasticity rules with the necessary anatomical and functional structures.
Although some aspects of neural reorganization involved in spontaneous recovery arise because of the unique biological state caused by injury, other aspects of neural reorganization that contribute to recovery relate to normal motor learning mechanisms [78, 79, 102]. It has been argued, in fact, that much of the recovery seen during the chronic phase of stroke is due to compensation, as defined in the introduction, which is enabled by motor learning . In this section, we briefly survey several models and features of motor learning relevant to computational neurorehabilitation models.
Forms of learning
There is a long-history of research in artificial intelligence linking different types of feedback to three forms of learning: unsupervised (or Hebbian) learning, supervised learning, and reinforcement learning, or, more simply paraphrased, learning features and representations, learning from errors, and learning from rewards . To learn a motor task, the learner needs feedback, exteroceptive, interoceptive, or both; as a result, in addition to unsupervised learning, supervised or reinforcement forms of learning are implicated in rehabilitation. Indeed, feedback, including the content and frequency of feedback, is known to modulate learning and rehabilitation efficacy [54, 55]. Different forms of learning have been associated with different biological substrates, although there is not a one-to-one mapping and the picture is still unfolding . For example, a form of unsupervised learning is LTP in pyramidal neurons . Supervised learning-like plasticity has been found to occur in the cerebellum [105, 106]. Some forms of reinforcement learning depend to a large extent on the dopaminergic system [107, 108], with the basal ganglia  and in particular the nucleus accumbens  also playing roles. Note that supervised learning is linked to the concept of Knowledge of Performance (KP) and reinforcement learning to the concept of Knowledge of Results (KR) in the motor learning and rehabilitation literature , although KP and KR likely also both act as reward signals for reinforcement learning, thus blurring this distinction.
Unsupervised learning is related to the concept of use-dependent learning, which refers to the phenomenon that the motor system can modify its performance through pure repetition of movements, without external feedback as to the success or failure of the movement [103, 111]. Several initial models of network dynamics after stroke incorporate unsupervised learning (see review ). Unsupervised learning, together with homeostatic plasticity, likely plays a role in map and neural reorganization post-stroke, and presumably in decreasing movement variability and thereby improving functional performance [89, 112].
Supervised learning and arm adaptation
A large number of studies of motor learning in the last 20 years have focused on elucidating aspects of supervised learning by observing the adaptation of arm movements in visuomotor rotations or force fields produced by robotic interfaces. It is still unclear how meaningful such studies of motor adaptation are for stroke rehabilitation, but they have inspired at least two innovative rehabilitation paradigms – error augmentation  and split-belt treadmill adaptation . Such studies have also generated some of the first and now fairly sophisticated mathematical models of motor learning, and thus serve as an example for how the development of computational models in stroke movement rehabilitation might proceed. Further, adaptation studies are relevant because individuals with a neurologic injury can adapt to predictable perturbations (e.g. [115, 116]), and likely continue to use motor adaptation to recalibrate limb systems in daily life (e.g. when they put on new shoes of a different weight, enter a swimming pool, or experience muscle fatigue ).
A key recent result is that a simple neural algorithm, which is a “sunken-v”, muscle-specific activation update law that relates the error experienced in muscle coordinates to the change in in muscle activation on the next movement trial, implements this minimization, while simultaneously shaping arm impedance to the task at hand [118–120]. Another factor involved in movement generation is that subjects tend to minimize time to complete an action, which stands in tradeoff with the required effort .
Learning occurs at multiple time scales as short as 10s of seconds [122, 123] and as long as several years . Multiple time-scales are also evident in the learning-performance distinction [102, 124], which will impact how models of recovery are structured. This sort of multiple time-constant dynamic also characterizes a broad range of motor learning literature encompassing a broader variety of tasks. Motor adaptation studies also shed light on multiple time scales, as motor adaptation occurs via simultaneous update of a fast process, which contributes to fast initial learning, and a slow process, which correlates with long-term retention ; these processes appear to be organized in parallel . Linear models with two time-constants implemented using a state space representation can account for a range of data on motor adaptation such as anterograde interference, spontaneous recovery, and savings under some conditions. Addition of non-linearities in the model allow for multiple adaptation and savings after washout [125, 126].
Whereas behavioral observations suggest that at least two learning processes are involved in adaptation, it is unclear how many distinct memories the brain actually updates. In addition, it is unclear whether these putative multiple motor memories reside within a single neural system that contains a distribution of possible timescales, or in qualitatively distinguishable neural systems. A recent study addressed these issues using a model-based fMRI approach . The behavioral data of subjects adapting to two opposing visuo-motor perturbations were first used to derive a large number of possible memory “states”, each with different dynamics, which were then correlated with neural activities. Regional specificity to timescales were identified. In particular, the activity in inferior parietal region and in the anterior-medial cerebellum was associated with memories for intermediate and long timescales, respectively. A sparse singular value decomposition analysis of variability in specificities to timescales over the brain identified four components, two fast, one middle, and one slow, each associated with different brain networks. Then, a multivariate decoding analysis showed that activity patterns in the anterior-medial cerebellum progressively represented the two rotations. These results thus support the existence of brain regions associated with multiple timescales in adaptation and a role of the cerebellum in storing multiple internal models.
Note that these multiple-time constant models assume error-based learning mechanisms. A recent summary of behavioral evidence concluded that while there are at least two components of motor adaptation in response to perturbations, they cannot be fully characterized by first order processes driven by error. For example, the slow process is implicit and learns form errors, while the fast process is explicit and is sensitive to success and failure, among other key differences . The evidence for reward-based and use-dependent mechanisms in motor adaptation suggest they operate at multiple time constants as well, and are likely to be of more relevance to restitution rather than compensation [129–131].
Such state-space models can account for short-term motor adaptation as well as multiple task learning and the contextual interference effect in post-stroke individuals [132, 133]. At least one initial computational neurorehabilitation model successfully used state-space models inspired by supervised learning data . In addition, robotically amplifying errors can help stroke patients eliminate steady state directional reaching errors . Further, training with amplified errors may increase arm movement recovery after chronic stroke . The beneficial effects of sensory augmentation may be due to the larger error available to the brain for perception and for learning. A recent study however suggests that augmenting errors can decrease motivation in a way that persists beyond the experience of the augmented errors , and motivation plays a key role in neurorehabilitation .
However, as mentioned above, the relevance of mechanisms of supervised learning of force fields and rotations to rehabilitation is limited, in part because rehabilitation, like motor skill learning, appears to be characterized by gradual improvement without a clear directional error signal, as when one tries to perform a fast movement or loses balance . This type of learning appears to be better characterized by reinforcement learning. Reinforcement learning theory  provides a framework for learning a “control policy” that maps states of the world to the actions that the agent should take in those states to maximize expected future rewards (or equivalently minimize future cost such as effort). Crucial to reward-based learning is the concept of exploration or search, which is necessary because there is no teacher. Instead, the learner must learn by trial and error. Compared to supervised learning, much less is known about how humans make use of reinforcement learning in learning motor behaviors; the issue has been explored to a larger extent in decision making. However, as we describe below, this search metaphor has been used successfully to simulate stroke rehabilitation, and replicate several key behavioral recovery observations [112, 136].
Note that both supervised and reinforcement learning likely operate simultaneously as both error and reward feedback are often available . For instance for fast reaching to targets by unimpaired subjects, it has been shown that different time constants of learning, and forgetting, may be associated with supervised and reinforcement learning . In rehabilitation therapy, receiving error feedback from a therapist can be rewarding and reinforces behavior. In the absence of external feedback, the learner still has access to intrinsic feedback and this can strongly promote self-learning. Thus, what is presumed to be unsupervised learning can be instead reinforcement learning driven by self-generated feedback. Also, it is the self-generated feedback that the patient needs to rely on when returning to his or her home environment. Accordingly, whenever external feedback is provided, it is important not to become too dependent on this source of augmented information by gradually weaning the learner from external feedback during practice, i.e. to learn to rely on self-generated feedback [137, 138].
Humans do not always appear to minimize error or maximize future rewards, however. In some instances, humans tend to perform a motor task by using the same strategy as they had used in previous trials, even if they had previously experienced a strategy using much less effort [139–142]. This suggests that rather than attempting global minimization of effort, the sensorimotor system might rather repeat a strategy that it knows will achieve the goal, a finding with implications for modeling use of compensatory movements by stroke patients.
Smoothness, generalization, and synergies
Several additional key aspects of motor learning that computational neurorehabilitation models will ultimately need to account for are the importance of sub-movements, generalization, and suppression of undesirable synergies. It is well known that the movements of individuals with stroke exhibit decreased smoothness. Variations in smoothness can be modeled as arising from patterns of stereotypical sub-movements, which may be neural “building blocks” that rehabilitation training must reassemble .
Generalization refers to the concept that training on one task can improve performance on other tasks. Patterns of generalization are complex, in that generalization has been found to be limited in some conditions , but rather broad in others . For example, after training to reach in one direction with a planar robotic perturbation, there is little transfer to other directions , but relatively broad generalization across certain arm postures . The concept that motor generalization is rather limited has helped drive a strong focus on task-specific training after stroke . However, a key qualifier of this concept is that the organization of practice may determine how much generalization occurs. If one trains one specific task or task variant, there may be little transfer. However if many task variants are practiced, transfer will likely be larger. This is called the variability of practice hypothesis , and has clear relevance for computational neurorehabilitation models.
Finally, motor learning not only involves building new action patterns but also suppressing or modulating pre-existing patterns or synergies. This is clear in bimanual skill learning in which the learner gradually overcomes the effect of pre-existing preferred coordination modes (such as in-phase and anti-phase patterns) that are part of the intrinsic dynamics of the system in order to acquire new coordination modes (such as less intrinsic, relative phase patterns) . Like any motor learner, individuals with a stroke may be more constrained by preferred coordination modes and/or by basic synergies that need to be overcome to develop skill. Similarly, previously acquired coordination modes can hamper the acquisition of new coordination modes, a phenomenon called negative transfer . Further, neural damage itself may fundamentally constrain the solutions possible for motor learning.
Outputs: functional outcomes and kinematics
Currently, in rehabilitation, behavior is usually described with relatively coarse scales, which often sum scores of performance on many tasks, and are typically taken at widely spaced time points. Ideally, computational neurorehabilitation models will bridge the causal link between network plasticity and behavioral changes, which will require higher resolution measurements of behavior at many repeated time points.
Note that data sets that evaluate outcomes differ from the data sets discussed in Section I A in that they quantify how much and how well a patient can move, rather than the total amount and features of rehabilitation training activity. There can be overlap between the two data sets, however, in that measurements made during training can be used to assess movement outcomes, and measurements made of movement during daily life can be used to quantify both training inputs (inasmuch as daily movement serves a training function), as well as serve as a way to quantify outcomes. For instance, kinematic measurements from a robotic rehabilitation device obtained during the course of robotic therapy have been shown to predict standard functional outcomes, without the need for dedicated assessment procedures .
Higher resolution outcomes data are becoming available through detailed kinematic studies of upper extremity movement in stroke recovery. In one study that serves as an example, patients with active proximal and distal limb movement within the first 2 weeks after stroke participating in the VECTORS trial were studied with kinematics and electromyography, identifying deficits in movement accuracy, reduced muscle efficiency, delayed muscle onsets, and a reduced ability to modulate muscle activity [150, 151]. Within the first 3 months after stroke, muscle onset times and percentage of muscle capabilities were similar to a neurologically-intact control group, but deficits in the ability to modulate muscle activity remained , including an inability to efficiently open and close the fingers on a target object [152, 153]. No computational neurorehabilitation models have yet to our knowledge attempted to model these outcomes.
Other longitudinal movement data from multiple labs around the world are accumulating [7, 154, 155]. For example, a recent kinematic study with intensive repeated measurements in the first months post stroke used principal components analysis to show that individuals with stroke learn to dissociate shoulder and elbow movements mainly in the early phase post-stroke, but do not achieve fully dissociated movements even at 26 weeks . Likewise, recovery in smoothness in reaching and hand aperture was mainly predicted by progress of time alone and almost plateaued within the first 8 weeks post stroke . Again, no models that incorporate plasticity mechanisms have yet attempted to model these findings.
Ideally, motion capture data sets would include the effect of different interventions. For example, there is an ongoing debate on the issue of whether recovery of functional movement is best achieved through restitution (such as reaching with normal kinematics) or compensation (such as using the less affected extremity or leaning forward with the trunk) [4, 157, 158]. At the present time, there are only small amounts of movement data collected pre- and post-intervention to address this issue. In a pilot trial of intensive, progressive, task-specific upper extremity training for people with stroke , kinematic and kinetic movement data were examined pre- and post-intervention to examine how movement changed . The results suggest that recovery of function via restitution versus compensation is not an all-or-none phenomenon, but varies within and across individuals. All patients demonstrated improvements in function on clinical scales. In contrast, some movement variables in some subjects indicated restitution of normal movement patterns, while other variables in the same or different subjects indicated the adoption of compensatory movement patterns .
Just as wearable sensors will drive computational neurorehabilitation models with data from self-training of the arm during home exercise or daily life, they will provide the descriptors of movement recovery that the models seeks to predict. Such sensors will provide data at a much finer temporal resolution than previous clinical data, which typically are obtained only at baseline, post-intervention, and at one or two follow-ups. This fact, along with the fact that the sensors provide kinematic data, will facilitate simulation of neural networks controlling movement recovery. Such technology-based measurements are also being found to map well to clinical outcome scales [149, 160–162]. Thus, these measures have validity in terms of established clinical measures, while enhancing the interpretation of these measures, facilitating more fine-grain modeling, and developing new measures. Further, sensor-based measures may catch improvements or differences in behavior when clinical assessment suffers from lack of resolution or floor/ceiling effects, e.g. . Again, we are at a propitious time for computational neurorehabilitation because of the rapid rise of new wearable sensing technologies, the data from which can be used to quantify functional outcomes important to patients in clinicians.
The current modeling benchmark: prognostic regression models
There is a rich body of work on statistical modeling of stroke recovery using regression models. A primary motivation for this work has been to develop prognostic models that support clinical decision making with regards to early stroke management, rehabilitation goals, and discharge planning . Such models take a “black-box” approach, seeking to identify the mapping between inputs, such as behavioral status and brain structure and function soon after stroke, and patient outcomes, such as long-term functional recovery. Such models can be driven solely by behavioral data combined with clinical descriptors of patients, by neurophysiological data (e.g. EMG, (f)MRI and Transcranial Magnetic Stimulation), or by both. These models usually do not have a priori hypotheses regarding the basis of this mapping in specific plasticity mechanisms, but they form a key benchmark against which the prognostic power of computational neurorehabilitation for clinical decision-making must be tested. That is, a key question is: “Will adding mechanistic details provide additional predictive power useful for clinical practice?” Accordingly, this section provides a brief overview of prognostic regression models.
Predicting outcome post stroke with baseline behavioral measures
Prospective cohort studies suggest that 33 to 66 % of stroke patients with a paretic upper limb do not show any recovery in upper limb function 6 months after stroke [164, 165]. In contrast, depending on the outcome measures used, 5 to 20 % will achieve full recovery of activities at 6 months. Multivariate regression models that are aimed to predict these outcomes are based on identifying variables (i.e. “markers” or “predictors”), usually measured at baseline, that are linearly or logistically associated with patient outcome at a later time post stroke. Baseline behavior markers that have been found to be useful in such models are voluntary movement ability measured at key joints, initial gross severity of stroke, initial disability, initial severity of motor deficits (e.g. Fugl-Meyer motor scores), and initial kinematics of reaching movements .
Two independently conducted prospective cohort studies showed that 98 % of individuals who preserve some voluntary finger extension and some shoulder abduction when assessed within the first 72 h post stroke regain some function at 6 months [20, 166, 167]. However, only 25 % of patients without voluntary control at 72 h regained some function at 6 months. The small proportion of false positives (≈2 %) and relatively large proportion of false negatives (≈25 %) in the SAFE model (Shoulder Abduction Finger Extension)  suggests that this clinical model may be too pessimistic in identifying patients who are likely not to recover meaningful function. The preservation or early return of some finger extension most likely reflects the necessity of some fibers of the corticospinal tract system in the affected hemisphere to remain intact in order to activate muscles of the forearm and hand [166, 168].
Numerous studies have also shown that the initial overall severity of stroke measured within 72 h after stroke onset, for example by using the NIH Stroke Scale  and initial disability, for example measured with the Barthel Index , are highly associated with the final outcome at 6 months measured with the NIH Stroke Scale, Barthel Index, or Functional Independence Measure. Retesting the Barthel Index at regular intervals significantly improved the model accuracy. These findings indicate that the timing of clinical assessment post stroke is an important factor that defines the accuracy of predicting final outcome .
Note this equation is a first order equation in FM. Thus, proportional recovery appears to describe a rule of spontaneous biological recovery that follows first order dynamics . In this framework, any early functional improvement that ensues is mainly happening via reductions in neurological impairment (i.e., restitution). Regression analyses of change scores in early started intensive repeated measurements post-stroke have shown that this time-window of spontaneous neurological recovery appears to be restricted to the first 3 months post stroke . Beyond this time, improvements in function appear to be mainly driven by compensation strategies in which patients learn to use their intact end-effectors to optimize motor performance [7, 79, 156, 175]. See also reviews about definitions, terminology  and phenomenology  of stroke recovery.
A recent development in prognostic modeling is the use of multiple input variables in non-linear regression models such as neural networks. Specifically, a set of kinematic measures assessed with a robotic device, then mapped through a nonlinear mapping algorithm, predicted clinical outcomes after stroke with higher precision than baseline clinical measures alone . This 30 min assessment asked patients to reach in the horizontal plane to visual targets, to draw circles, and to move against robotic resistance. Metrics included deviation from a straight line, aim, average speed, peak speed, movement duration, smoothness of movement, and features of the sub-movements of movement trajectories, including number, duration, overlap, peak, interval, and skewness of sub-movements. Model performance plateaued when using about eight of these features in the predictive model; that is, additional features did not improve prediction, presumably because of high correlations between some of these features. The nonlinearity of the model was essential to its improved predictive power, which suggests that successful computational neurorehabilitation models will incorporate nonlinearities as well. Note that this model, like “classical” regression models, is purely statistical, and does not explicitly model brain reorganization or learning following stroke.
Predicting outcome post-stroke with brain imaging measures
Another way to look at proportional recovery is that anatomical damage sets a limit on the extent of possible recovery by restitution and therapy takes advantage of plasticity mechanisms (whether enhanced during the early period of spontaneous biological recovery or normal in the chronic phase) to help patients achieve the maximum recovery possible. Predictive models thus need to include neuro-anatomical variables . However, given that level of initial severity is the best predictor of final outcome, we need to ask whether brain imaging data can improve model performance. In other words, we are interested in the factors that determine recovery over and above those that cause impairment, as these are different things .
Residual structural and functional architecture after stroke can also be used to estimate clinical outcome. Diffusion tensor imaging (DTI) is able to assess integrity of white matter tracts and when measured within 3 weeks of subcortical stroke, corticospinal tract (CST) integrity correlates with both initial and 6 month upper limb impairment . In a separate study, damage to the CST at the posterior limb of the internal capsule 12 h post-stroke correlated well with motor impairment at 30 and 90 days . Stinear and colleagues have proposed an algorithm for sequentially combining simple clinical, TMS, and DTI measures to predict upper limb function . The PREP (Predicting REcovery Potential) algorithm was tested in a sample of 40 sub-acute stroke patients and performed well in predicting motor function based on Action Research Arm Test scores at 12 weeks post-stroke. The performance of DTI in this setting should be improved by making the tracts specific to particular functions e.g. upper limb  and developing ways for the assessment of tract integrity to be done in a standardised  and automatic  manner.
Corticospinal tract integrity correlates with initial upper limb severity, which explains why it often correlates with final outcome. However, it remains unclear whether it can explain outcomes over and above initial impairment. The role of intact cortical regions in supporting motor recovery is unknown, but has been explored in language recovery. Predicting language outcome and recovery after stroke (PLORAS) uses the whole structural brain scan from which voxel-wise estimates of the likelihood of damaged tissue are derived . This ‘lesion-map’ for each patient is added to (i) time since stroke and (ii) a detailed assessment of various language capabilities. Using a machine learning approach, a new subject's lesion image is compared with those from all the other patients already in the database to find one with a similar lesion. The language scores for all the similar patients are plotted over time, enabling the time course of recovery for the new patient to be estimated . The potential for such an approach extends to many domains including motor and cognitive outcomes. Using this type of neuroimaging complex biomarker discovery  it should be possible to provide accurate prognostic models allowing accurate goal setting in neurorehabilitation and stratification in clinical trials . Note that for goal setting in neurorehabilitation, for any patients for whom a given model predicts little potential benefit of treatment, future research will hopefully reveal new, modifiable factors that can be targeted for such patients.
Multivariate machine learning approaches have also been applied to functional MRI data to predict outcome. For example, fMRI data acquired within 2 weeks of stroke in patients with aphasia was used to predict outcome in language . Accuracy in predicting good and bad outcome at 6 months was 76 % and improved to 86 % when age and baseline language impairment were added to the classification model. In the motor domain, fMRI data acquired in the first few days after stroke has been used to try to predict a subsequent change in motor performance . Using a multivariate analysis, a specific pattern of activated voxels was identified as highly predictive of clinical change over the subsequent 3 months, a finding that was independent of initial stroke severity and lesion volume. Anatomical hypotheses could not be tested using this multivariate approach – the study simply indicated that predictive signal was present in a pattern of activation.
Predicting treatment effects
Predicting outcome will be useful for clinical and research stratification, but what a clinician would like to know is what are the chances of a patient responding to a specific intervention. Stinear and colleagues  set out to determine whether characterising the state of the motor system would help in predicting an individual patient's capacity for further functional improvement at least 6 months post-stroke in a subsequent motor practice programme. A combination of TMS, structural and functional MRI was used to suggest a method for determining who would respond to training. This approach has also been used to predict likely response to robotic training, with both structural  and functional imaging  data making some contribution.
When it comes to assessing the effects of treatments thought to enhance the potential for experience dependent plasticity, less work has been done. Currently, there is a problematic explanatory gap between molecular (from animal studies) and behavioral (from human studies) accounts of the mechanisms of recovery after stroke. Lack of progression of knowledge from animal models to benefit for human stroke patients has led to the search for ways to study these mechanisms in human subjects. There are exciting advances in how human neuroimaging data can be analyzed that suggest a way forward. Specifically, it is now possible to examine changes in organization of the human brain after stroke at multiple levels of brain architecture, ranging from large-scale networks to alterations in synaptic physiology.
A number of analytical approaches are available to examine connectivity within well-defined macroscopic brain networks. For example, graph theory can be used to determine ‘efficiency’ of information transfer around small world networks such as the brain . This allows inferences to be made about functional connectivity at the level of whole brain, hemisphere or specified network level and can be applied to fMRI, EEG, and MEG data. Dynamic Causal Modeling (DCM) is a method of analyzing data from a dynamic system such as the brain. Bayesian inversion of a specified anatomical model given the empirical data allows the determination of model parameters, which reflect effective connectivity between brain regions (i.e. the task-dependent influence of one brain region on another) . DCM of induced responses acquired with MEG  is particularly appealing as it allows partitioning of the effective coupling between regions at the same spectral frequency (linear coupling) and across frequencies (non-linear coupling). Non-linear coupling in particular is important in functional integration .
Another recently developed approach to studying brain dynamics after stroke is to use patient-specific structural connectivity data obtained from MRI to set-up an individualized Virtual Brain model of the patient. By optimizing model parameters, such as long-range coupling or local inhibition, in order to match resting state BOLD signals, insight can be gained into how those parameters vary with different types of stroke .
At the mesoscopic scale, the spectral characteristics of brain oscillations measured with MEG (or EEG) in the gamma and beta frequency are dependent on the balance of activity between populations of excitatory (glutamatergic) pyramidal cells and inhibitory (GABAergic) interneurons  and are candidate biomarkers of the potential for both local- and network-plasticity . More recently, it has become possible to define plausible biophysical DCMs to examine mesoscopic interactions between populations of excitatory and inhibitory cells in specific cortical regions using data from MEG . This approach has been validated using local field potentials in animal preparations where independent pharmacological/microdialysis assays have served to corroborate modelling results . A recent example of how DCM of canonical microcircuits can provide mechanistic inferences is the finding that psilocybin, a 5HT-2A agonist, reduced oscillatory power in posterior association cortex by increasing excitability of deep-layer pyramidal neurons .
This range of neuroimaging tools and computational approaches will provide the appropriate intermediate level of description with which to bridge the gap between what we know about recovery after stroke from animal models compared to what we know from studies of behavior in humans. A more detailed knowledge of how these processes are related to impairment and recovery following stroke will provide a mechanistic framework for understanding how to treat patients more effectively. It will open the way for functional brain imaging to become a clinically useful tool in rehabilitation, particularly for our ability to predict outcomes and response to novel plasticity enhancing therapies.
In summary, there is a rich history and promising future to using behavioural status and brain structure and function soon after stroke, to predict long-term motor recovery. Prognostic regression models can inform computational neurorehabilitation models and are a key benchmark against which the predictive power of computational neurorehabilitation will need to be tested.
Computational neurorehabilitation models
Computational neurorehabilitation models discussed in this review
Model A: Han et al. 2008 
Structure: A bilateral limb-use model using a population vector framework and reinforcement and error-based learning.
Example Prediction: If spontaneous recovery, motor training, or both, bring function above a certain threshold, then training can be stopped, as the repeated spontaneous arm use provides a form of motor learning that further bootstraps function and spontaneous use (i.e. the “virtuous cycle”)
Model B: Reinkensmeyer et al. 2012 
Structure: A wrist strength recovery model using a simplified corticospinal neural network and reinforcement learning via stochastic search
Example Prediction: Reinforcement learning can explain a broad range of features of stroke recovery, including exponential recovery, residual capacity, and shift of brain activation to secondary motor areas.
Model C: Casadio and Sanguineti, 2012 
Structure: An arm impairment reduction model using a linear, discrete-time, shift invariant dynamical system driven by data from robotic therapy
Example Prediction: A parameter describing retention predicts Fugl-Meyer score 3 months following robotic therapy.
Model D: Hidaka et al. 2012 
Structure: First order dynamic model that incorporates a modifiable parameter that controls the effect of arm function on use.
Example Prediction: Therapy increased the parameter that controls the effect of arm function on use. An increase in this parameter, which can be thought of as the confidence to use the arm for a given level of function, led to an increase in spontaneous use after therapy compared to before therapy.
Model E: Reinkensmeyer 2003 
Structure: Adaptive Markov model with Hebbian plasticity that maps relationship between normal and abnormal sensory and motor states, allowing for physical assistance from a rehabilitation trainer
Example Prediction: Assistance-as-needed can enhance recovery beyond what is possible with unassisted movement practice.
Model F: Jarrassé et al. 2012 
Structure: Uses a cost function with error and effort terms, generated by both the therapist (or robot) and human trainee, to characterize a broad range of interactive behaviors of two-agent systems.
Example prediction: Sensorimotor rehabilitation may be modeled in terms of the cost functions that the trainee and the trainer seek to implement, as well as the algorithms they use to implement those cost functions.
Reaching the threshold for recovery in bilateral hand use
A key aspect of stroke motor recovery is that individuals with a stroke can elect not to use their impaired arm, since they usually have a relatively unaffected arm to perform most needed tasks (i.e. they can compensate). This “learned non-use” may logically be expected to contribute to loss of motor control of the hemiparetic arm (just as an athlete or musician who stops practicing becomes rusty), although to our knowledge there is no research that has yet documented this loss. Han et al. developed one of the first computational neurorehabilitation models to study the interactions between adaptive decision making related to learned non-use and motor relearning after a simulated motor cortex lesion  (Fig. 1c). The inputs to the model are targets for bilateral reaching practice and the outputs are the choice of arm to use to reach to the given target, and the kinematic accuracy of the reaching movement. The model incorporates a reward-based learning mechanism for arm selection, and an error-based learning mechanism for refining the neural population code in primary motor cortex that specifies reach direction.
One clinical impact of the model is that it suggests a novel therapeutic paradigm, the “Train-Wait-Train” paradigm, that not only would test the threshold hypothesis, but also which could potentially be a more cost-effective way to deliver rehabilitation therapy. For example, in computer simulations, a virtual patient was first given a set of 200 trials of therapy, simulated by forced use of the affected arm. Then the patient experienced repeated cycles of 1000 trials in a free arm choice condition (wait), followed by 100 treatment trials (train). The initial training trials were not sufficient to reach threshold, and spontaneous use stayed low. After further training sessions, however, the simulated patient entered the “virtuous circle.” Experiments are currently testing this Train-Wait-Train paradigm.
The model also provides insights into the time constants of stroke motor recovery. The main recovery time constant was on the order of thousands of trials. However, the time constant controlling the change in the decision to use one hand or the other was much smaller, and as a result, the model developed learned non-use soon after stroke. Because how fast learned non-use actually develops after stroke is unknown, hand choice and kinematic data collected soon after stroke would lead to better parameter estimation and hence better predictions from these models.
This model, however, has a number of limitations that need to be addressed to better understand and predict individual recovery as a function of use and motor training. For example, the model is simply bi-stable: either the patient is “above threshold” and fully recovers, or below and does not. In addition, the model ignores interactions between the two cortices. An extension of this model, which includes inter-hemispheric inhibition, has been proposed to account for the beneficial effects of bi-manual training compared to uni-manual training . Further, the model is only concerned with recovery of the control of movement direction, with no attention paid to arm muscle activity, other kinematic features, or functional upper extremity behaviors.
Recovering from weakness via reinforcement learning
Motor recovery after stroke is characterized by a seemingly disparate set of behavioral and brain imaging observations, many reviewed above, but could these observations arise from a few fundamental features of human sensorimotor plasticity? Reinkensmeyer et al. approached this question by focusing on the modeling of the recovery of distal upper limb strength, the rationale being that strength strongly predicts upper extremity functional activity [136, 201]. The inputs to the model are attempts to flex the wrist, and the output is the flexion force achieved. For the motor system to recover wrist flexion strength, the model assumes that the motor system must learn appropriate activation of residual corticospinal cells (i.e. those preserved after the stroke – thus, within the terminology adopted in this paper this is again a study of neural compensation not restitution) via repeated movement attempts followed by reward feedback. A simplified corticospinal network model was developed based on several key observations from primate neurophysiological experiments – that corticospinal cell activity sums to create a net excitatory flexor or extensor drive to a joint, that different corticospinal cells have different “gains” for exciting motoneuronal pools, and that the relationship between a corticospinal cell’s activity and its individual contribution to muscle force is linear up to a peak firing rate, then saturates for higher activity levels. Finally, the model assumed that the key underlying mechanism of plasticity driving strength recovery after stroke was a reinforcement learning mechanism, in which the sensorimotor system modifies corticospinal cell activations based on repetitive movement experiences, using stochastic random search, such that limb force output is maximized.
This model predicts a broad range of the features of stroke recovery. It predicts the dose-response curve of rehabilitation, and putative modifiers of this curve, such as that the same dose of exercise has a smaller effect when given to a more severely impaired patient, and that the same dose of exercise given early has a larger differential effect than when given later. This latter finding suggests that increased sub-acute plasticity may arise in part due to normative, compensatory network learning dynamics rather than solely as a function of the stroke-altered tissue microenvironment.
Further, and unlike the Han et al. model reviewed above, the model predicts exponential-like strength recovery curves that never quite reach an asymptote, but instead exhibit a residual capacity for further recovery with further movement practice, as has been observed experimentally . At the level of structural brain imaging observations, the model predicts that patients with a larger residual corticospinal network will recover more, a known stroke phenomenon . It also predicts a key functional imaging observation, which is that movement-related activation in secondary motor areas increases following a stroke that damages output from the primary motor area .
Mathematically, two key mechanisms drive the observed network dynamics. First, the saturation in the individual relationships between corticospinal cell firing rates and their contribution to muscle force slows later learning, thus leaving room for residual capacity (Fig. 4d). This is because force results from the summed contributions of many cells. Once the activations of a subset of these cells have evolved in the correct direction to maximize force output, and become saturated, then those neurons can no longer contribute to further increases in force. As saturated neurons randomly vary their output from trial to trial, they exhibit an increased probability of decreasing the force produced on subsequent trials, since they can now only vary their output in the direction counterproductive to further force increase. Second, the activations of more strongly connected cells tend to be modified first, as changes in their activations are more likely to increase force output (Fig. 4c).
A key clinical implication of the model is that, if a single learning mechanism – reinforcement learning via stochastic search – can explain such a broad range of features of stroke recovery, then perhaps this mechanism should be targeted in rehabilitation training to facilitate the search process. Adding variability and chances for exploration to training regimens and providing augmented feedback on the teaching signal are two possible ways to achieve this; a large literature on reinforcement learning in computer science may also be brought to bear on rehabilitation protocol optimization (see above). The model predicts another unexpected way that an improved search might be achieved, an “Inhibit-Train-Release” paradigm, which is to temporarily inhibit cells that are already optimized (perhaps using a localized inhibition scheme, such as infused muscimol or focused transcranial magnetic stimulation, for example), train the remaining cells, and then release the inhibited cells. Stochastic search tends to optimize cells with stronger effects on muscles, and once optimized these cells block further optimization as explained above. The model predicts that temporarily inhibiting optimized cells will allow non-optimized cell activations to become optimized.
As discussed above, this model successfully accounts for a large number of existing neural and behavioral data. However, it has several limitations. It has cartoon-like complexity, modeling only a single flexor and extensor muscle, and simplifying corticospinal activation to static input-output relationships. In addition, it does not model spontaneous recovery or restitution, as we have defined them here. Further, while severe weakness precludes control, mild to moderate weakness is less well correlated with motor control. For example, the ipsilateral limb after stroke shows abnormal motor control but strength is normal , so motor control requires more than strength.
Robot assistance, retention, and learning predicts recovery
Models such as the Han et al. model or the Reinkensmeyer model above are “qualitative”, i.e., they make general qualitative predictions of recovery. Parameters in these models are “hand-tuned”. As such, they are only loosely based on actual data and cannot be used to predict recovery of individual patients. As reviewed above, during rehabilitative training mediated by a robot it is possible to observe and record the patient’s performance as well as the level of assistance that the patient received. Can the trial-to-trial evolution in movement success during rehabilitation be modeled, and, if so, what are the key learning parameters needed to describe this evolution? Casadio et al.  developed the first computational model driven by data from actual robotic training sessions, which describes the trial-by-trial evolution of the recovery process induced by robotic training. This model provides insights into the role of assistive force in the recovery process, and the extent to which learned changes in voluntary control decay over time and transfer to subsequent training sessions.
The model characterizes the recovery process related to robot-assisted training for improving arm extension in chronic stroke survivors as a linear, discrete-time, shift invariant dynamical system. The model posits that motor performance is a function of a voluntary control component, an assistive force component, provided either by a robot or a therapist, and a performance noise term. Thus, the level of physical assistance provided during rehabilitative training is taken as one of the system’s inputs. The voluntary control component is the internal state of the model and its temporal evolution is described as the combination of three additive terms: a ‘memory’ or ‘retention term’, accounting for how much voluntary control depends on previous experience; an ‘assistance’ component expressed as the magnitude of the assistive force; and a learning component that accounts for how much voluntary control on the actual trial is affected by the previous value of the driving signal. The input signals that drive recovery are movement error or a performance measure. A process noise term accounts for the portion of the recovery not due to these three terms.
The most striking result from this model is that the retention parameter predicts the percent change of the Fugl-Meyer score at the 3-month point following the end of the robotic treatment (Fig. 4b). This result, which needs to be confirmed with a larger cohort, is potentially important for the individualization of training. After one or few initial training sessions with the robot, the model can be fitted to the data. Then, by examining the retention parameter, one could potentially determine who will benefit from additional robotic motor training. Note however that this model is based on a set up in which the arm is constrained to move in two dimensions with shoulder and elbow movements. Thus, here again, for example, the issue of restitution versus compensation is difficult to address and the conclusions are limited to simplified movements at only two joints.
Understanding interactions between function and use
In the same vein of using computational models for predicting individual recovery, Schweighofer and colleagues developed a first-order dynamical model of stroke recovery with longitudinal data from participants receiving constraint-induced movement therapy in the EXCITE clinical trial . The goal of the model was to better understand the interactions between arm function and use in human post-stroke following therapy. The model shows the existence of time-varying interactions between self-reported amount of daily arm use and recovery of function via a dynamical model of stroke recovery in the 2 years following therapy. The time constants in this model were on the order of several weeks. Of most importance, by comparing the model parameters before and after constraint-induced movement therapy intervention in participants receiving the intervention 1 year after randomization, therapy increased the parameter that controls the effect of arm function on use. An increase in this parameter, which can be thought of as the confidence to use the arm for a given level of function, led to an increase in self-reported spontaneous use after therapy compared to before therapy. Thus, here, the parameters of the model can be viewed as “states” that have physiological meaning, which may expedite the testing of new experimental hypotheses.
However, as in the previous model of Casadio et al., this is a highly simplified model of recovery. It contains only a few parameters and two time constants, one for functional recovery and the other for hand use. The reason for this simplicity is the very sparse data set on which the model was built. Increasing the complexity of the model, and thus the number of parameters was not possible because this would have led to overfitting, i.e., the model would fit the data very well but would not generalize.
Modeling the effect of assistance-as-needed
Rehabilitation therapy often involves interactions between patient and a trainer, be it a rehabilitation therapist, a robotic device, or a computer game. Modeling the patient-trainer interaction could potentially provide insight into movement recovery.
Reinkensmeyer used an adaptive Markov model to examine the role of external mechanical assistance from a robotic device or therapist in promoting movement recovery . Following research in gait training after spinal cord injury, the model assumes that motor control is characterized by repeated transitions between sensory and motor states; for example, sensing of full hip extension at the end of stance triggers leg flexion. It further grossly characterizes these states as abnormal and normal sensory states (e.g. appropriate or inappropriate hip extension), and abnormal and normal motor states (effective or ineffective leg flexion). The model uses a Hebbian-inspired model of plasticity in which the transitions between specific sensory and specific motor states become more reliable with repetitive activation of that transition. The action of a skilled external trainer (robot or human) who is assisting in movement is modeled as a mediated increase in the probability of transferring to a state of normal motor output (e.g. the trainer helps generate hip extension sensory inputs to enable effective leg flexion). Assistance-as-needed is simulated by mediating this transfer only when the patient is in an abnormal motor state, while assistance-always is simulated by mediating this increase on every movement repetition.
The model predicts that assistance-as-needed can enhance recovery beyond what is possible with unassisted practice, that assistance-always is not as effective as assistance-as-needed, that the trainer’s skill in assisting toward normal motor output matters in reinforcing normal state transitions, and that assistance is not useful when sensory input is less directly coupled to motor output. While these predictions may sound somewhat intuitive, this is perhaps because they mirror opinions that are often expressed in the current clinical milieu. The model in this case served to verify that these opinions could be mathematically supported relying on a simple but plausible plasticity rule.
Patient-trainer dynamics as an optimization
We contend that the models reviewed in the previous section are evidence that a fundamental understanding of neurologic recovery will be facilitated by modeling motor learning and plasticity itself in a context specific to rehabilitation. Even the initial, simplified models presented are generating novel ideas concerning the mechanisms of recovery in rehabilitation, and thus are suggesting important directions for future experimental research.
Novel ideas from initial models and their limitations
For example, the Han et al. model discussed above suggests the existence of a threshold of movement ability that allows patients to enter a “virtuous” cycle of activity-dependent plasticity; this premise is now being tested in a clinical trial. The Reinkensmeyer et al. model suggests that a large number of existing behavioral and neural data can be accounted for if the motor system uses a reinforcement-learning paradigm – stochastic search – to optimize neural activations; this suggests increasing research into techniques to enhance reinforcement learning in order to enhance rehabilitation therapy, a proposition that is only beginning to be explored. The Casadio et al. model showed that a short-term measure of retention in robotic training can accurately predict how much long-term recovery is possible with such training during chronic stroke, suggesting a novel assay for patient selection for robotic therapy. Other interesting ideas generated by the initial models reviewed above are that a key way that rehabilitation therapy may function is by modulating a parameter that controls the effect of arm function on use, and that trainer-trainee interactions can be characterized using parsimonious cost functions.
Even given their utility in generating such novel ideas concerning rehabilitation, these initial models clearly have shortcomings. They greatly simplify human movement and the rehabilitation process. They ignore the fact that every day activity relies on bimanual motor function. They selectively model only one or two learning and plasticity mechanisms. They focus on motor learning rather than plasticity mechanisms associated with restitution, as yet neglecting the rich literature on spontaneous biological recovery, which may ultimately offer greater potential for recovery. They have not yet been used to replicate, let alone extend, the findings achieved with prognostic regression models. None have yet used a cross-validation procedure in order to see how much a model based on the data from a pool of patients can explain the behavior of patients outside of that pool. In addition, sensorimotor control is just one of many domains addressed by neurorehabilitation.
The forthcoming data revolution
Another key argument that this review has attempted to make is that the time is propitious for continued development of computational neurorehabilitation models. As we surveyed above, the mathematical tools for developing such models are already at least partly available because of the past several decades of work in the field of computational neuroscience and machine learning. Further, as we also reviewed above, large-scale longitudinal data for each patient and for a large number of patients are now possible to obtain with robotic devices and wearable sensors. This will allow the development of more elaborate, physics-based models that predict recovery at a fine temporal and spatial resolution. Note that conventional motor learning experiments have generally failed to even come close to the number of training trials that gives rise to the high level expertise in sports and work-related skills. With instrumented rehabilitation and wearable sensors, this situation is now set to change, at least in the context of motor learning during rehabilitation. A key challenge for the future is how to utilize these longitudinal data that are collected on different samples, with different methods of motion capture, and different quantifications of movement into a coherent package such that they can inform models of movement control over the course of recovery after stroke.
The Casadio et al. model described above initiates the data-driven approach, as it is the first to use trial-to-trial data from actual robotic training to predict changes in Fugl-Meyer scores, suggesting that retention-related parameters correlate with impairment recovery. Following this approach, we predict that soon we will see the emergence of large scale mechanistic models of motor recovery that are driven by the actual movement content of rehabilitation therapy as well as records of daily activity. Ideally, one would use the same data set from hundreds or even thousands of well-characterized patients, so that computational model output is realistic and appropriate.
In addition, the models will increasingly be informed by automatic analysis of MRI scans routinely obtained after stroke, since, as we reviewed above, brain anatomical information is needed because identical behavior can arise through largely different neural processes. A range of novel neuroimaging tools and computational methods, including analysis of grey and white matter structures and structural and functional connectivity , will provide an intermediate level of description with which to bridge the gap between what we know about recovery after stroke from animal models compared to what we know from studies of behavior in humans.
Complexification and utility
Clearly, initial computational neurorehabilitation models are vast simplifications of a very complex process. While simplification and abstraction are often virtues in modeling, with richer data sets, it will be possible to increase the complexity of computational neurorehabilitation models with less risk of overfitting. Models containing a multiple joint system, such as the whole arm or both arms, will be important for understanding compensation. Upper extremity motor control during real-world, free-living activity involves the movement of all the segments in order to position and orient the limb and to interact with objects [213, 214]. If a computational model utilizes only a few of the segments, then the model output will provide only a limited view of the actual solution, essentially ignoring the degrees of freedom problem underscored by Bernstein. Models that can help understand or predict control processes during naturalistic actions will be of high value for the field of neurorehabilitation.
Further, it is currently difficult to experimentally study the interacting effects between different forms of learning, such as supervised, unsupervised, and reinforcement learning, including the role of the timing of rehabilitation on these processes [81, 215]. There appear to be methods also to induce beneficial plasticity beyond task repetition, such as the possible enhancing of effects of non-invasive brain stimulation on motor learning [216, 217]. Computational neurorehabilitation models can incorporate multiple levels of plasticity and learning, as well as plasticity-enhancing effects of techniques such as electrical stimulation, and even psychological effects important to rehabilitation, helping understand these interactions in computer simulations to guide future experimental work. This is important, as one theoretically could vet hypotheses in an efficient and cost-effective manner, rather than relying solely on randomized, controlled trials, which are costly and time consuming.
A key question, of course, is whether the incorporation of plasticity and learning mechanisms, along with internal physiological states, into models will improve upon the predictive capability now possible with prognostic regression models. We contend they have a good chance to, because they are more likely to isolate the key predictive variables of interest, since these variables likely relate to physiologic function, and computational neurorehabilitation models seek to make just such variables explicit. Such variables likely vary from patient-to-patient as well, suggesting that their isolation will improve individualized predictions.
If so, computational models of neurorehabilitation should ultimately improve rehabilitation for individuals with neurologic injuries. We expect that computational models of recovery, based on early clinical data, kinematic performance, and routine scans, will provide the basis for future clinical software that suggests timing, dosage, and content of therapy. Such an approach will transform neurorehabilitation by guiding clinicians, patients, and health providers in the optimization of treatments.
The authors would like to thank the Borchard Foundation for supporting this work.
DJR was supported by grant NIH R01 HD062744
EB was supported by grants EU-FP7 ICT-601003 BALANCE, ICT-2013-10 SYMBITRON, and EU-H2020 ICT-644727 COGIMON.
CEL was supported by grant NIH R01 HD068290
MC was supported by grant FP7-PEOPLE-2012-CIG-334201 (REMAKE)
NS was supported by grant NIH R01 HD065438.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Feigin VL, Forouzanfar MH, Krishnamurthi R, Mensah GA, Connor M, Bennett DA, Moran AE, Sacco RL, Anderson L, Truelsen T, O’Donnell M, Venketasubramanian N, Barker-Collo S, Lawes CMM, Wang W, Shinohara Y, Witt E, Ezzati M, Naghavi M, Murray C. Global and regional burden of stroke during 1990–2010: findings from the Global Burden of Disease Study 2010. Lancet. 2014;383(9913):245–55.PubMedPubMed CentralView ArticleGoogle Scholar
- Miller EL, Murray L, Richards L, Zorowitz RD, Bakas T, Clark P, Billinger SA. Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: a scientific statement from the American Heart Association. Stroke. 2010;41(10):2402–48.PubMedView ArticleGoogle Scholar
- Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet. 2011;377(9778):1693–702.PubMedView ArticleGoogle Scholar
- Levin MF, Kleim JA, Wolf SL. What do motor ‘recovery’ and ‘compensation’ mean in patients following stroke? Neurorehabil Neural Repair. 2009;23(4):313–9.PubMedView ArticleGoogle Scholar
- Langhorne P, Coupar F, Pollock A. Motor recovery after stroke: a systematic review. Lancet Neurol. 2009;8(8):741–54.PubMedView ArticleGoogle Scholar
- Buma F, Kwakkel G, Ramsey N. Understanding upper limb recovery after stroke. Restor Neurol Neurosci. 2013;31(6):707–22.PubMedGoogle Scholar
- van Kordelaar J, van Wegen EEH, Nijland RHM, Daffertshofer A, Kwakkel G. Understanding adaptive motor control of the paretic upper limb early poststroke: the EXPLICIT-stroke program. Neurorehabil Neural Repair. 2013;27(9):854–63.PubMedView ArticleGoogle Scholar
- Kitago T, Goldsmith J, Harran M, Kane L, Berard J, Huang S, Ryan SL, Mazzoni P, Krakauer JW, Huang VS. Robotic therapy for chronic stroke: general recovery of impairment or improved task-specific skill? J Neurophysiol. 2015;114(3):1885–94.PubMedView ArticleGoogle Scholar
- Lohse KR, Lang CE, Boyd LA. Is more better? Using metadata to explore dose-response relationships in stroke rehabilitation. Stroke. 2014;45(7):2053–8.PubMedPubMed CentralView ArticleGoogle Scholar
- Krakauer JW, Carmichael ST, Corbett D, Wittenberg GF. Getting neurorehabilitation right: what can be learned from animal models? Neurorehabil Neural Repair. 2012;26(8):923–31.PubMedPubMed CentralView ArticleGoogle Scholar
- Milot MH, Spencer SJ, Chan V, Allington JP, Klein J, Chou C, Bobrow JE, Cramer SC, Reinkensmeyer DJ. Re-training upper extremity movement with the robotic exoskeleton BONES after chronic stroke: a crossover pilot study evaluating the functional outcomes of more naturalistic robotic movement training. J Neuroeng Rehabil. 2013;10:112.PubMedPubMed CentralView ArticleGoogle Scholar
- Awad LN, Reisman DS, Pohlig RT, and S. A. Binder-Macleod. Reducing The Cost of Transport and Increasing Walking Distance After Stroke: A Randomized Controlled Trial on Fast Locomotor Training Combined With Functional Electrical Stimulation. Neurorehabil. Neural Repair. 2015. pii: 1545968315619696. [Epub ahead of print]Google Scholar
- Clark DJ, Neptune RR, Behrman AL, Kautz SA. Locomotor Adaptability Task Promotes Intense and Task-Appropriate Output From the Paretic Leg During Walking. Arch Phys Med Rehabil. 2016;97(3):493–6.PubMedView ArticleGoogle Scholar
- Jones TA, Adkins DL. Motor System Reorganization After Stroke: Stimulating and Training Toward Perfection. Physiology (Bethesda). 2015;30(5):358–70.Google Scholar
- Chollet F, Tardy J, Albucher J-F, Thalamas C, Berard E, Lamy C, et al. Fluoxetine for motor recovery after acute ischaemic stroke (FLAME): a randomised placebo-controlled trial. Lancet Neurol. 2011;10(2):123–30.Google Scholar
- Ng KL, Gibson EM, Hubbard R, Yang J, Caffo B, O’Brien RJ, Krakauer JW, Zeiler SR. Fluoxetine Maintains a State of Heightened Responsiveness to Motor Training Early After Stroke in a Mouse Model. Stroke. 2015;46(10):2951–60.PubMedView ArticleGoogle Scholar
- Janssen H, Bernhardt J, Collier JM, Sena ES, McElduff P, Attia J, Pollack M, Howells DW, Nilsson M, Calford MB, Spratt NJ. An enriched environment improves sensorimotor function post-ischemic stroke. Neurorehabil Neural Repair. 2010;24(9):802–13.PubMedView ArticleGoogle Scholar
- Tiozzo E, Youbi M, Dave K, Perez-Pinzon M, Rundek T, Sacco RL, Loewenstein D, Lewis JE, Wright CB. Aerobic, Resistance, and Cognitive Exercise Training Poststroke. Stroke. 2015;46(7):2012–6.PubMedView ArticleGoogle Scholar
- Veerbeek JM, Kwakkel G, van Wegen EEH, Ket JCF, Heymans MW. Early Prediction of Outcome of Activities of Daily Living After Stroke: A Systematic Review. Stroke. 2011;42(5):1482–8.PubMedView ArticleGoogle Scholar
- Kwakkel G, Kollen BJ. Predicting activities after stroke: what is clinically relevant? Int J Stroke. 2013;8(1):25–32.PubMedView ArticleGoogle Scholar
- Robertson IH, Murre JM. Rehabilitation of brain damage: brain plasticity and principles of guided recovery. Psychol Bull. 1999;125(5):544–75.PubMedView ArticleGoogle Scholar
- Goodall S, Reggia JA, Chen Y, Ruppin E, Whitney C. A computational model of acute focal cortical lesions. Stroke. 1997;28(1):101–9.PubMedView ArticleGoogle Scholar
- Reggia JA. Neurocomputational models of the remote effects of focal brain damage. Med Eng Phys. 2004;26(9):711–22.PubMedView ArticleGoogle Scholar
- Levitan S, Reggia JA. Interhemispheric effects on map organization following simulated cortical lesions. Artif Intell Med. 1999;17(1):59–85.PubMedView ArticleGoogle Scholar
- Casadio M, Tamagnone I, Summa S, Sanguineti V. Neuromotor recovery from stroke: computational models at central, functional, and muscle synergy level. Front Comput Neurosci. 2013;7:97.PubMedPubMed CentralView ArticleGoogle Scholar
- Aimone JB, Weick JP. Perspectives for computational modeling of cell replacement for neurological disorders. Front Comput Neurosci. 2013;7:150.PubMedPubMed CentralView ArticleGoogle Scholar
- Kwakkel G, van Peppen R, Wagenaar RC, Wood Dauphinee S, Richards C, Ashburn A, Miller K, Lincoln N, Partridge C, Wellwood I, Langhorne P. Effects of augmented exercise therapy time after stroke: a meta-analysis. Stroke. 2004;35(11):2529–39.PubMedView ArticleGoogle Scholar
- Langhorne P, Wagenaar R, Partridge C. Physiotherapy after stroke: more is better? Physiother Res Int. 1996;1(2):75–88.PubMedView ArticleGoogle Scholar
- Veerbeek JM, Koolstra M, Ket JCF, van Wegen EEH, Kwakkel G. Effects of augmented exercise therapy on outcome of gait and gait-related activities in the first 6 months after stroke: a meta-analysis. Stroke. 2011;42(11):3311–5.PubMedView ArticleGoogle Scholar
- Veerbeek JM, van Wegen E, van Peppen R, van der Wees PJ, Hendriks E, Rietberg M, Kwakkel G. What is the evidence for physical therapy poststroke? A systematic review and meta-analysis. PLoS One. 2014;9(2):e87987.PubMedPubMed CentralView ArticleGoogle Scholar
- Krakauer JW. The applicability of motor learning to neurorehabilitation. In: Oxford Textbook of Neurorehabilitation. 2015. p. 55.Google Scholar
- Tan WY, Ye Z. Estimation of HIV infection and incubation via state space models. Math Biosci. 2000;167(1):31–50.PubMedView ArticleGoogle Scholar
- Cazelles B, Chau NP. Using the Kalman filter and dynamic models to assess the changing HIV/AIDS epidemic. Math Biosci. 1997;140(2):131–54.PubMedView ArticleGoogle Scholar
- Venkatakrishnan K, Friberg LE, Ouellet D, Mettetal JT, Stein A, Trocóniz IF, Bruno R, Mehrotra N, Gobburu J, Mould DR. Optimizing oncology therapeutics through quantitative translational and clinical pharmacology: challenges and opportunities. Clin Pharmacol Ther. 2015;97(1):37–54.PubMedView ArticleGoogle Scholar
- Emken JL, Reinkensmeyer DJ. Robot-enhanced motor learning: accelerating internal model formation during locomotion by transient dynamic amplification. IEEE Trans Neural Sys Rehab Eng. 2005;13(1):33–9.View ArticleGoogle Scholar
- Wolf SL, Kwakkel G, Bayley M, McDonnell MN. Best practice for arm recovery post stroke: an international application. Physiotherapy. 2016;102(1):1–4.PubMedView ArticleGoogle Scholar
- Ramos-Murguialday A, Broetz D, Rea M, Läer L, Yilmaz O, Brasil FL, Liberati G, Curado MR, Garcia-Cossio E, Vyziotis A, Cho W, Agostini M, Soares E, Soekadar S, Caria A, Cohen LG, Birbaumer N. Brain-machine interface in chronic stroke rehabilitation: a controlled study. Ann Neurol. 2013;74(1):100–8.PubMedPubMed CentralView ArticleGoogle Scholar
- Bernhardt J, Chan J, Nicola I, Collier JM. Little therapy, little physical activity: rehabilitation within the first 14 days of organized stroke unit care. J Rehabil Med. 2007;39(1):43–8.PubMedView ArticleGoogle Scholar
- Harris JE, Eng JJ, Miller WC, Dawson AS. A self-administered Graded Repetitive Arm Supplementary Program (GRASP) improves arm function during inpatient stroke rehabilitation: a multi-site randomized controlled trial. Stroke. 2009;40(6):2123–8.PubMedView ArticleGoogle Scholar
- Lang CE, Macdonald JR, Reisman DS, Boyd L, Jacobson KT, Schindler-Ivens SM, et al. Observation of amounts of movement practice provided during stroke rehabilitation. Arch Phys Med Rehabil. 2009;90(10):1692–8.Google Scholar
- Birkenmeier RL, Prager EM, Lang CE. Translating animal doses of task-specific training to people with chronic stroke in 1-hour therapy sessions: a proof-of-concept study. Neurorehabil Neural Repair. 2010;24(7):620–35.PubMedPubMed CentralView ArticleGoogle Scholar
- Waddell KJ, Birkenmeier RL, Moore JL, Hornby TG, Lang CE. Feasibility of high-repetition, task-specific training for individuals with upper-extremity paresis. Am J Occup Ther. 2014;68(4):444–53.PubMedPubMed CentralView ArticleGoogle Scholar
- Park H, Kim S, Winstein CJ, Gordon J, Schweighofer N. Short-Duration and Intensive Training Improves Long-Term Reaching Performance in Individuals With Chronic Stroke. Neurorehabil Neural Repair. 2015. pii: 1545968315606990. [Epub ahead of print].Google Scholar
- Turolla A, Dam M, Ventura L, Tonin P, Agostini M, Zucconi C, Kiper P, Cagnin A, Piron L. Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial. J Neuroeng Rehabil. 2013;10:85.PubMedPubMed CentralView ArticleGoogle Scholar
- Molina KI, Ricci NA, de Moraes SA, Perracini MR. Virtual reality using games for improving physical functioning in older adults: a systematic review. J Neuroeng Rehabil. 2014;11(1):156.PubMedPubMed CentralView ArticleGoogle Scholar
- van Diest M, Lamoth CJC, Stegenga J, Verkerke GJ, Postema K. Exergaming for balance training of elderly: state of the art and future developments. J Neuroeng Rehabil. 2013;10:101.PubMedPubMed CentralView ArticleGoogle Scholar
- Kwakkel G, Kollen BJ, Krebs HI. Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair. 2008;22(2):111–21.PubMedPubMed CentralView ArticleGoogle Scholar
- Levin MF, Weiss PL, Keshner EA. Emergence of Virtual Reality as a Tool for Upper Limb Rehabilitation: Incorporation of Motor Control and Motor Learning Principles. Phys Ther. 2014;95(3):415–25.PubMedPubMed CentralView ArticleGoogle Scholar
- Reinkensmeyer DJ, Boninger ML. Technologies and combination therapies for enhancing movement training for people with a disability. J Neuroeng Rehabil. 2012;9:17.PubMedPubMed CentralView ArticleGoogle Scholar
- Balasubramanian S, Klein J, Burdet E. Robot-assisted rehabilitation of hand function. Curr Opin Neurol. 2010;23(6):661–70.PubMedView ArticleGoogle Scholar
- Maciejasz P, Eschweiler J, Gerlach-Hahn K, Jansen-Troy A, Leonhardt S. A survey on robotic devices for upper limb rehabilitation. J Neuroeng Rehabil. 2014;11:3.PubMedPubMed CentralView ArticleGoogle Scholar
- Basteris A, Nijenhuis SM, Stienen AHA, Buurke JH, Prange GB, Amirabdollahian F. Training modalities in robot-mediated upper limb rehabilitation in stroke: a framework for classification based on a systematic review. J Neuroeng Rehabil. 2014;11:111.PubMedPubMed CentralView ArticleGoogle Scholar
- Chen CC, Bode RK. Factors influencing therapist’s decision-making in the acceptance of new technology devices in stroke rehabilitation. Am J Phys Med Rehabil. 2011;90:415–25.PubMedView ArticleGoogle Scholar
- Schmidt RA. Motor control and learning: A behavioral emphasis. Champaign, Il USA: Human Kinetics Publishers; 2005.Google Scholar
- Giggins OM, Persson UM, Caulfield B. Biofeedback in rehabilitation. J Neuroeng Rehabil. 2013;10:60.PubMedPubMed CentralView ArticleGoogle Scholar
- Casadio M, Sanguineti V. Learning, retention, and slacking: a model of the dynamics of recovery in robot therapy. IEEE Trans Neural Syst Rehabil Eng. 2012;20(4):286–96.PubMedView ArticleGoogle Scholar
- Reinkensmeyer DJ, Maier MA, Guigon E, Chan V, Akoner OM, Wolbrecht ET, Cramer SC, Bobrow JE. Do Robotic and Non-Robotic Arm Movement Training Drive Motor Recovery after Stroke by a Common Neural Mechanism? Experimental Evidence and a Computational Model. Proc IEEE Eng Med Biol Conf. 2009;2009:2439–41.Google Scholar
- Ellis MD, Sukal-Moulton TM, Dewald JPA. Impairment-Based 3-D Robotic Intervention Improves Upper Extremity Work Area in Chronic Stroke: Targeting Abnormal Joint Torque Coupling With Progressive Shoulder Abduction Loading. IEEE Trans Robot. 2009;25(3):549–55.PubMedPubMed CentralView ArticleGoogle Scholar
- Lum PS, Burgar CG, Kenney DE, Van der Loos HF. Quantification of force abnormalities during passive and active-assisted upper-limb reaching movements in post-stroke hemiparesis. Ieee Trans Biomed Eng. 1999;46(6):652–62.PubMedView ArticleGoogle Scholar
- Lum PS, Burgar CG, Shor PC. Evidence for improved muscle activation patterns after retraining of reaching movements with the MIME robotic system in subjects with post-stroke hemiparesis. IEEE Trans Neural Syst Rehabil Eng. 2004;12(2):186–94.PubMedView ArticleGoogle Scholar
- Galvez JA, Budovitch A, Harkema SJ, Reinkensmeyer DJ. Trainer variability during step training after spinal cord injury: Implications for robotic gait training device design. J Rehabil Res Dev. 2011;48(2):147–59.PubMedView ArticleGoogle Scholar
- Formaggio E, Storti SF, Boscolo Galazzo I, Gandolfi M, Geroin C, Smania N, Spezia L, Waldner A, Fiaschi A, Manganotti P. Modulation of event-related desynchronization in robot-assisted hand performance: brain oscillatory changes in active, passive and imagined movements. J Neuroeng Rehabil. 2013;10(1):24.PubMedPubMed CentralView ArticleGoogle Scholar
- Zimmermann R, Marchal-Crespo L, Edelmann J, Lambercy O, Fluet M-C, Riener R, Wolf M, Gassert R. Detection of motor execution using a hybrid fNIRS-biosignal BCI: a feasibility study. J Neuroeng Rehabil. 2013;10(1):4.PubMedPubMed CentralView ArticleGoogle Scholar
- Gassert R, Burdet E, Chinzei K. Opportunities and challenges in MR-compatible robotics: reviewing the history, mechatronic components, and future directions of this technology. IEEE Eng Med Biol Mag. 2008;27(3):15–22.PubMedView ArticleGoogle Scholar
- Schweighofer N, Han CE, Wolf SL, Arbib MA, Winstein CJ. A functional threshold for long-term use of hand and arm function can be determined: predictions from a computational model and supporting data from the Extremity Constraint-Induced Therapy Evaluation (EXCITE) Trial. Phys Ther. 2009;89(12):1327–36.PubMedPubMed CentralView ArticleGoogle Scholar
- Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil. 2012;9:21.PubMedPubMed CentralView ArticleGoogle Scholar
- Steins D, Dawes H, Esser P, Collett J. Wearable accelerometry-based technology capable of assessing functional activities in neurological populations in community settings: a systematic review. J Neuroeng Rehabil. 2014;11:36.PubMedPubMed CentralView ArticleGoogle Scholar
- van der Lee JH, Beckerman H, Knol DL, de Vet HCW, Bouter LM. Clinimetric properties of the Motor Activity Log for the assessment of arm use in hemiparetic patients. Stroke. 2004;35(6):1404–10.View ArticleGoogle Scholar
- Lang CE, Wagner JM, Edwards DF, Dromerick AW. Upper Extremity Use in People with Hemiparesis in the First Few Weeks After Stroke. J Neurol Phys Ther. 2007;31(2):56–63.PubMedView ArticleGoogle Scholar
- Bailey RR, Birkenmeier RL, Lang CE. Real-world affected upper limb activity in chronic stroke: an examination of potential modifying factors. Top Stroke Rehabil. 2015;22:26–33.PubMedPubMed CentralView ArticleGoogle Scholar
- Uswatte G, Miltner WH, Foo B, Varma M, Moran S, Taub E. Objective measurement of functional upper-extremity movement using accelerometer recordings transformed with a threshold filter. Stroke. 2000;31(3):662–7.PubMedView ArticleGoogle Scholar
- Rand D, Eng JJ. Predicting daily use of the affected upper extremity 1 year after stroke. J Stroke Cerebrovasc Dis. 2015;24(2):274–83.PubMedView ArticleGoogle Scholar
- van der Pas SC, Verbunt JA, Breukelaar DE, van Woerden R, Seelen HA. Assessment of arm activity using triaxial accelerometry in patients with a stroke. Arch Phys Med Rehabil. 2011;92(9):1437–42.PubMedView ArticleGoogle Scholar
- Bailey RR, Klaesner JW, Lang CE. Quantifying Real-World Upper-Limb Activity in Nondisabled Adults and Adults With Chronic Stroke. Neurorehabil Neural Repair. 2015;29(10):969–78.PubMedView ArticleGoogle Scholar
- Swinnen SP. Intermanual coordination: from behavioural principles to neural-network interactions. Nat Rev Neurosci. 2002;3(5):348–59.PubMedView ArticleGoogle Scholar
- Kortier HG, Sluiter VI, Roetenberg D, Veltink PH. Assessment of hand kinematics using inertial and magnetic sensors. J Neuroeng Rehabil. 2014;11:70.PubMedPubMed CentralView ArticleGoogle Scholar
- N. Friedman, J. B. Rowe, D. J. Reinkensmeyer, M. Bachman. The manumeter: A wearable device for monitoring daily use of the wrist and fingers. J Biomed Heal Informatics. epub ahead, 2014.Google Scholar
- Krakauer JW. Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr Opin Neurol. 2006;19(1):84–90.PubMedView ArticleGoogle Scholar
- Kitago T, Krakauer JW. Motor learning principles for neurorehabilitation. Handb Clin Neurol. 2013;110:93–103.PubMedView ArticleGoogle Scholar
- Huang VSC, Krakauer JW. Robotic neurorehabilitation: a computational motor learning perspective. J Neuroeng Rehabil. 2009;6(1):5.PubMedPubMed CentralView ArticleGoogle Scholar
- Murphy TH, Corbett D. Plasticity during stroke recovery: from synapse to behaviour. Nat Rev Neurosci. 2009;10(12):861–72.PubMedView ArticleGoogle Scholar
- Duncan PW, Goldstein LB, Matchar D, Divine GW, Feussner J. Measurement of motor recovery after stroke. Stroke. 1992;23:1084–9.PubMedView ArticleGoogle Scholar
- Feldman DE, Brecht M. Map plasticity in somatosensory cortex. Science. 2005;310(5749):810–5.PubMedView ArticleGoogle Scholar
- Clarkson AN, Huang BS, Macisaac SE, Mody I, Carmichael ST. Reducing excessive GABA-mediated tonic inhibition promotes functional recovery after stroke. Nature. 2010;468(7321):305–9.PubMedPubMed CentralView ArticleGoogle Scholar
- Schiene K, Bruehl C, Zilles K, Qü M, Hagemann G, Kraemer M, Witte OW. Neuronal hyperexcitability and reduction of GABAA-receptor expression in the surround of cerebral photothrombosis. J Cereb Blood Flow Metab. 1996;16(5):906–14.PubMedView ArticleGoogle Scholar
- Qü M, Mittmann T, Luhmann HJ, Schleicher A, Zilles K. Long-term changes of ionotropic glutamate and GABA receptors after unilateral permanent focal cerebral ischemia in the mouse brain. Neuroscience. 1998;85(1):29–43.PubMedView ArticleGoogle Scholar
- Liepert J, Storch P, Fritsch A, Weiller C. Motor cortex disinhibition in acute stroke. Clin Neurophysiol. 2000;111(4):671–6.PubMedView ArticleGoogle Scholar
- Witte OW, Buchkremer-Ratzmann I, Schiene K, Neumann-Haefelin T, Hagemann G, Kraemer M, Zilles K, Freund HJ. Lesion-induced network plasticity in remote brain areas. Trends Neurosci. 1997;20(8):348–9.PubMedView ArticleGoogle Scholar
- Bains AS, Schweighofer N. Time-sensitive reorganization of the somatosensory cortex poststroke depends on interaction between Hebbian and homeoplasticity: a simulation study. J Neurophysiol. 2014;112(12):3240–50.PubMedPubMed CentralView ArticleGoogle Scholar
- Schmidt A, Wellmann J, Schilling M, Strecker J-K, Sommer C, Schäbitz W-R, Diederich K, Minnerup J. Meta-analysis of the efficacy of different training strategies in animal models of ischemic stroke. Stroke. 2014;45(1):239–47.PubMedView ArticleGoogle Scholar
- Cramer SC, Chopp M. Recovery recapitulates ontogeny. Trends Neurosci. 2000;23(6):265–71.PubMedView ArticleGoogle Scholar
- Li S, Overman JJ, Katsman D, Kozlov SV, Donnelly CJ, Twiss JL, Giger RJ, Coppola G, Geschwind DH, Carmichael ST. An age-related sprouting transcriptome provides molecular control of axonal sprouting after stroke. Nat Neurosci. 2010;13(12):1496–504.PubMedPubMed CentralView ArticleGoogle Scholar
- Nudo RJ, Wise BM, SiFuentes F, Milliken GW. Neural substrates for the effects of rehabilitative training on motor recovery after ischemic infarct. Science (80-). 1996;272:1791–4.View ArticleGoogle Scholar
- Zeiler SR, Krakauer JW. The interaction between training and plasticity in the poststroke brain. Curr Opin Neurol. 2013;26(6):609–16.PubMedPubMed CentralView ArticleGoogle Scholar
- Nudo RJ. Recovery after damage to motor cortical areas. Curr Opin Neurobiol. 1999;9(6):740–7.PubMedView ArticleGoogle Scholar
- Lytton WW, Stark JM, Yamasaki DS, Sober SJ. Computer models of stroke recovery: implications for neurorehabilitation. Neuroscientist. 1999;5:100–11.View ArticleGoogle Scholar
- Sober SJ, Stark JM, Yamasaki DS, Lytton WW. Receptive field changes after strokelike cortical ablation: a role for activation dynamics. J Neurophysiol. 1997;78(6):3438–43.PubMedGoogle Scholar
- Alstott J, Breakspear M, Hagmann P, Cammoun L, Sporns O. Modeling the impact of lesions in the human brain. PLoS Comput Biol. 2009;5(6):e1000408.PubMedPubMed CentralView ArticleGoogle Scholar
- Honey CJ, Sporns O. Dynamical consequences of lesions in cortical networks. Hum Brain Mapp. 2008;29(7):802–9.PubMedView ArticleGoogle Scholar
- Rubinov M, McIntosh AR, Valenzuela MJ, Breakspear M. Simulation of neuronal death and network recovery in a computational model of distributed cortical activity. Am J Geriatr Psychiatry. 2009;17(3):210–7.PubMedView ArticleGoogle Scholar
- Reinkensmeyer DJ, Iobbi MG, Kahn LE, Kamper DG, Takahashi CD. Modeling reaching impairment after stroke using a population vector model of movement control that incorporates neural firing rate variability. Neural Comput. 2003;15(11):2619–42.PubMedView ArticleGoogle Scholar
- Winstein CJ. Knowledge of results and motor learning--implications for physical therapy. Phys Ther. 1991;71(2):140–9.PubMedGoogle Scholar
- Wolpert DM, Diedrichsen J, Flanagan JR. Principles of sensorimotor learning. Nat Rev Neurosci. 2011;12:739–51.PubMedGoogle Scholar
- Rioult-Pedotti MS, Friedman D, Donoghue JP. Learning-induced LTP in neocortex. Science (80-). 2000;290(5491):533–6.View ArticleGoogle Scholar
- Schweighofer N, Doya K, Kuroda S. Cerebellar aminergic neuromodulation: towards a functional understanding. Brain Res Brain Res Rev. 2004;44(2–3):103–16.PubMedView ArticleGoogle Scholar
- Kawato M, Kuroda S, Schweighofer N. Cerebellar supervised learning revisited: biophysical modeling and degrees-of-freedom control. Curr Opin Neurobiol. 2011;21(5):791–800.PubMedView ArticleGoogle Scholar
- Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol. 1998;80(1):1–27.PubMedGoogle Scholar
- Doya K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr Opin Neurobiol. 2000;10(6):732–9.PubMedView ArticleGoogle Scholar
- Fee MS, Goldberg JH. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience. 2011;198:152–70.PubMedPubMed CentralView ArticleGoogle Scholar
- Kelley AE, Smith-Roe SL, Holahan MR. Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core. Proc Natl Acad Sci U S A. 1997;94(22):12174–9.PubMedPubMed CentralView ArticleGoogle Scholar
- Bains A, Schweighofer N. “Robust Use-Dependent Learning in Arm Movements,” TCMC proceedings, Chicago. 2015. Accessed at: http://www.seas.harvard.edu/motorlab/acmc/openconf.php.
- Han CE, Arbib MA, Schweighofer N. Stroke Rehabilitation Reaches a Threshold. PLoS Comput Biol. 2008;4(8):e1000133.PubMedPubMed CentralView ArticleGoogle Scholar
- Abdollahi F, Case ED, Listenberger M, Kenyon RV, Kovic M, Bogey R, Hedeker D, Jovanovic B, Patton JL. Error augmentation enhancing arm recovery in individuals with chronic hemiparetic stroke: a randomized crossover design. Neurorehabili Neural Repair (NNR). 2014;28(2):120–8.View ArticleGoogle Scholar
- Reisman DS, Wityk R, Silver K, Bastian AJ. Locomotor adaptation on a split-belt treadmill can improve walking symmetry post-stroke. Brain. 2007;130(Pt 7):1861–72.PubMedPubMed CentralView ArticleGoogle Scholar
- Takahashi CD, Reinkensmeyer DJ. Hemiparetic stroke impairs anticipatory control of arm movement. Exp Brain Res. 2003;149(2):131–40.PubMedGoogle Scholar
- Patton JL, Kovic M, Mussa-Ivaldi FA. Custom-designed haptic training for restoring reaching ability to individuals with poststroke hemiparesis. J Rehabil Res Dev. 2006;43(5):643–56.PubMedView ArticleGoogle Scholar
- Takahashi CD, Nemet D, Rose-Gottron CM, Larson JK, Cooper DM, Reinkensmeyer DJ. Effect of muscle fatigue on internal model formation and retention during reaching with the arm. J Appl Physiol. 2006;100(2):695–706.PubMedView ArticleGoogle Scholar
- Emken JL, Benitez R, Sideris A, Bobrow JE, Reinkensmeyer DJ. Motor adaptation as a greedy optimization of error and effort. J Neurophysiol. 2007;97:3997–4006.PubMedView ArticleGoogle Scholar
- Franklin DW, Burdet E, Tee KP, Osu R, Chew CM, Milner TE, Kawato M. CNS learns stable, accurate, and efficient movements using a simple algorithm. J Neurosci. 2008;28(44):11165–73.PubMedView ArticleGoogle Scholar
- Franklin DW, Osu R, Burdet E, Kawato M, Milner TE. Adaptation to stable and unstable dynamics achieved by combined impedance control and inverse dynamics model. J Neurophysiol. 2003;90:3270–82.PubMedView ArticleGoogle Scholar
- Rigoux L, Guigon E. A model of reward- and effort-based optimal decision making and motor control. PLoS Comput Biol. 2012;8(10):e1002716.PubMedPubMed CentralView ArticleGoogle Scholar
- Smith MA, Ghazizadeh A, Shadmehr R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 2006;4(6):e179.PubMedPubMed CentralView ArticleGoogle Scholar
- Kording KP, Tenenbaum JB, Shadmehr R. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat Neurosci. 2007;10(6):779–86.PubMedPubMed CentralView ArticleGoogle Scholar
- Cahill L, McGaugh JL, Weinberger NM. The neurobiology of learning and memory: some reminders to remember. Trends Neurosci. 2001;24(10):578–81.PubMedView ArticleGoogle Scholar
- Lee J-Y, Schweighofer N. Dual adaptation supports a parallel architecture of motor memory. J Neurosci. 2009;29(33):10396–404.PubMedPubMed CentralView ArticleGoogle Scholar
- Zarahn E, Weston GD, Liang J, Mazzoni P, Krakauer JW. Explaining savings for visuomotor adaptation: linear time-invariant state-space models are not sufficient. J Neurophysiol. 2008;100(5):2537–48.PubMedPubMed CentralView ArticleGoogle Scholar
- S. S. Kim, K. Ogawa, J. Lv, N. Schweighofer, H. Imamizu. Multiple time constants in sensorimotor adaptation: a model-based fMRI study. PLOS Biol. in press, 2015.Google Scholar
- Huberdeau DM, Krakauer JW, Haith AM. Dual-process decomposition in human sensorimotor adaptation. Curr Opin Neurobiol. 2015;33:71–7.PubMedView ArticleGoogle Scholar
- Huang V, Haith A, Mazzoni P, Krakauer J. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron. 2011;70:787–801.PubMedPubMed CentralView ArticleGoogle Scholar
- Shmuelof L, Huang VS, Haith AM, Delnicki RJ, Mazzoni P, Krakauer JW. Overcoming motor ‘forgetting’ through reinforcement of learned actions. J Neurosci. 2012;32(42):14617–21.PubMedPubMed CentralView ArticleGoogle Scholar
- Galea JM, Mallia E, Rothwell J, Diedrichsen J. The dissociable effects of punishment and reward on motor learning. Nat Neurosc. 2015;18:597–602.View ArticleGoogle Scholar
- Schweighofer N, Lee J-Y, Goh H-T, Choi Y, Kim SS, Stewart JC, Lewthwaite R, Winstein CJ. Mechanisms of the contextual interference effect in individuals poststroke. J Neurophysiol. 2011;106(5):2632–41.PubMedPubMed CentralView ArticleGoogle Scholar
- Scheidt RA, Stoeckmann T. Reach adaptation and final position control amid environmental uncertainty after stroke. J Neurophysiol. 2007;97(4):2824–36.PubMedView ArticleGoogle Scholar
- Duarte JE, Reinkensmeyer DJ. Effects of robotically modulating kinematic variability on motor skill learning and motivation. J Neurophysiol. 2015;113(7):2682–91.PubMedPubMed CentralView ArticleGoogle Scholar
- Sutton RS, Barto AG. Reinforcement learning: An introduction. Cambridge: MIT Press; 1998.Google Scholar
- Reinkensmeyer DJ, Guigon E, Maier MA. A computational model of use-dependent motor recovery following stroke: optimizing corticospinal activations via reinforcement learning can explain residual capacity and other strength recovery dynamics. Neural Netw. 2012;29–30:60–9.PubMedView ArticleGoogle Scholar
- Izawa J, Shadmehr R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol. 2011;7(3):e1002012.PubMedPubMed CentralView ArticleGoogle Scholar
- Salmoni AW, Schmidt RA, Walter CB. Knowledge of results and motor learning: a review and critical reappraisal. Psychol Bull. 1984;95(3):355–86.PubMedView ArticleGoogle Scholar
- Ganesh G, Haruno M, Kawato M, Burdet E. Motor memory and local minimization of error and effort, not global optimization, determine motor behavior. J Neurophysiol. 2010;104(1):382–90.PubMedView ArticleGoogle Scholar
- de Rugy A, Loeb GE, Carroll TJ. Muscle coordination is habitual rather than optimal. J Neurosci. 2012;32(21):7384–91.PubMedView ArticleGoogle Scholar
- Kistemaker DA, Wong JD, Gribble PL. The cost of moving optimally: kinematic path selection. J Neurophysiol. 2014;112(8):1815–24.PubMedPubMed CentralView ArticleGoogle Scholar
- Kistemaker DA, Wong JD, Gribble PL. The central nervous system does not minimize energy cost in arm movements. J Neurophysiol. 2010;104(6):2985–94.PubMedView ArticleGoogle Scholar
- Rohrer B, Fasoli S, Krebs HI, Hughes R, Volpe B, Frontera WR, et al. Movement smoothness changes during stroke recovery. J Neurosci. 2002;22(18):8297–304.Google Scholar
- Gandolfo F, Mussa-Ivaldi FA, Bizzi E. Motor learning by field approximation. Proc Natl Acad Sci U S A. 1996;93(9):3843–6.PubMedPubMed CentralView ArticleGoogle Scholar
- Shadmehr R, Mussa-Ivaldi FA. Adaptive representation of dynamics during learning of a motor task. J Neurosci. 1994;14(5):3208–24.PubMedGoogle Scholar
- French B, Thomas LH, Leathley MJ, Sutton CJ, McAdam J, Forster A, Langhorne P, Price CI, Walker A, Watkins CL. Repetitive task training for improving functional ability after stroke. Cochrane Database Syst Rev. 2007;4(4):CD006073.PubMedGoogle Scholar
- Schmidt RA. A schema theory of discrete motor skill learning. Psychol Rev. 1975;82(4):225–60.View ArticleGoogle Scholar
- Brashers-Krug T, Shadmehr R, Bizzi E. Consolidation in human motor memory. Nature. 1996;382(6588):252–5.PubMedView ArticleGoogle Scholar
- Zariffa J, Kapadia N, Kramer JL, Taylor P, Alizadeh-Meghrazi M, Zivanovic V, Albisser U, Willms R, Townson A, Curt A, Popovic MR, Steeves JD. Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury. IEEE Trans Neural Syst Rehabil Eng. 2012;20(3):341–50.PubMedView ArticleGoogle Scholar
- Dromerick AW, Lang CE, Birkenmeier RL, Wagner JM, Miller JP, Videen TO, Powers WJ, Wolf SL, Edwards DF. Very Early Constraint-Induced Movement during Stroke Rehabilitation (VECTORS): A single-center RCT. Neurology. 2009;73(3):195–201.PubMedPubMed CentralView ArticleGoogle Scholar
- Wagner JM, Dromerick AW, Sahrmann SA, Lang CE. Upper extremity muscle activation during recovery of reaching in subjects with post-stroke hemiparesis. Clin Neurophysiol. 2007;118(1):164–76.PubMedPubMed CentralView ArticleGoogle Scholar
- Lang CE, Wagner JM, Bastian AJ, Hu Q, Edwards DF, Sahrmann SA, Dromerick AW. Deficits in grasp versus reach during acute hemiparesis. Exp Brain Res. 2005;166(1):126–36.PubMedView ArticleGoogle Scholar
- Lang CE, DeJong SL, Beebe JA. Recovery of thumb and finger extension and its relation to grasp performance after stroke. J Neurophysiol. 2009;102(1):451–9.PubMedPubMed CentralView ArticleGoogle Scholar
- Cheung VCK, Turolla A, Agostini M, Silvoni S, Bennis C, Kasi P, Paganoni S, Bonato P, Bizzi E. Muscle synergy patterns as physiological markers of motor cortical damage. Proc Natl Acad Sci U S A. 2012;109(36):14652–6.PubMedPubMed CentralView ArticleGoogle Scholar
- Ellis MD, Kottink AIR, Prange GB, Rietman JS, Buurke JH, Dewald JPA. Quantifying loss of independent joint control in acute stroke with a robotic evaluation of reaching workspace. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:8231–4.PubMedGoogle Scholar
- van Kordelaar J, van Wegen E, Kwakkel G. Impact of time on quality of motor control of the paretic upper limb after stroke. Arch Phys Med Rehabil. 2014;95(2):338–44.PubMedView ArticleGoogle Scholar
- Michaelsen SM, Dannenbaum R, Levin MF. Task-specific training with trunk restraint on arm recovery in stroke: randomized control trial. Stroke. 2006;37(1):186–92.PubMedView ArticleGoogle Scholar
- Roby-Brami A, Feydy A, Combeaud M, Biryukova EV, Bussel B, Levin MF. Motor compensation and recovery for reaching in stroke patients. Acta Neurol Scand. 2003;107(5):369–81.PubMedView ArticleGoogle Scholar
- DeJong SL, Birkenmeier RL, Lang CE. Person-specific changes in motor performance accompany upper extremity functional gains after stroke. J Appl Biomech. 2012;28(3):304–16.PubMedPubMed CentralGoogle Scholar
- Del Din S, Patel S, Cobelli C, Bonato P. Estimating Fugl-Meyer clinical scores in stroke survivors using wearable sensors. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:5839–42.PubMedGoogle Scholar
- Friedman N, Chan V, Reinkensmeyer AN, Beroukhim A, Zambrano G, Bachman M, Reinkensmeyer DJ. Retraining and assessing hand movement after stroke using the MusicGlove: Comparison with conventional hand therapy and isometric grip training. J Neuroeng Rehabil Res. 2014;11:76.View ArticleGoogle Scholar
- Krebs HI, Krams M, Agrafiotis DK, DiBernardo A, Chavez JC, Littman GS, Yang E, Byttebier G, Dipietro L, Rykman A, McArthur K, Hajjar K, Lees KR, Volpe BT. Robotic measurement of arm movements after stroke establishes biomarkers of motor recovery. Stroke. 2014;45(1):200–4.PubMedPubMed CentralView ArticleGoogle Scholar
- van Dokkum L, Hauret I, Mottet D, Froger J, Métrot J, Laffont I. The contribution of kinematics in the assessment of upper limb motor recovery early after stroke. Neurorehabil Neural Repair. 2014;28(1):4–12.PubMedView ArticleGoogle Scholar
- Sunderland A, Tinson DJ, Bradley EL, Fletcher D, Langton Hewer R, Wade DT. Enhanced physical therapy improves recovery of arm function after stroke. A randomized controlled trial. J Neurol Neurosurgery Psychiatry. 1992;55:530–5.View ArticleGoogle Scholar
- Wade DT, Langton-Hewer R, Wood VA, Skilbeck CE, Ismail HM. The hemiplegic arm after stroke: measurement and recovery. J Neurol Neurosurg Psychiatry. 1983;46(6):521–4.PubMedPubMed CentralView ArticleGoogle Scholar
- Stinear C. Prediction of recovery of motor function after stroke. Lancet Neurol. 2010;9(12):1228–32.PubMedView ArticleGoogle Scholar
- Nijland RHM, van Wegen EEH, Harmeling-van der Wel BC, Kwakkel G. Presence of finger extension and shoulder abduction within 72 hours after stroke predicts functional recovery: early prediction of functional outcome after stroke: the EPOS cohort study. Stroke. 2010;41(4):745–50.PubMedView ArticleGoogle Scholar
- Stinear CM, Barber PA, Petoe M, Anwar S, Byblow WD. The PREP algorithm predicts potential for upper limb recovery after stroke. Brain A J Neurol. 2012;135(Pt 8):2527–35.View ArticleGoogle Scholar
- Kwakkel G, Veerbeek JM, van Wegen EEH, Nijland R, Harmeling-van der Wel BC, Dippel DWJ. Predictive value of the NIHSS for ADL outcome after ischemic hemispheric stroke: does timing of early assessment matter? J Neurol Sci. 2010;294(1–2):57–61.PubMedView ArticleGoogle Scholar
- Kwakkel G, Veerbeek JM, Harmeling-van der Wel BC, van Wegen E, Kollen BJ. Diagnostic accuracy of the Barthel Index for measuring activities of daily living outcome after ischemic hemispheric stroke: does early poststroke timing of assessment matter? Stroke. 2011;42(2):342–6.PubMedView ArticleGoogle Scholar
- Prabhakaran S, Zarahn E, Riley C, Speizer A, Chong JY, Lazar RM, Marshall RS, Krakauer JW. Inter-individual variability in the capacity for motor recovery after ischemic stroke. Neurorehabil Neural Repair. 2007;22(1):64–71.PubMedView ArticleGoogle Scholar
- Zarahn E, Alon L, Ryan SL, Lazar RM, Vry M-S, Weiller C, Marshall RS, Krakauer JW. Prediction of motor recovery using initial impairment and fMRI 48 h poststroke. Cereb Cortex (New York, NY 1991). 2011;21(12):2712–21.View ArticleGoogle Scholar
- Winters C, van Wegen EEH, Daffertshofer A, Kwakkel G. Generalizability of the Proportional Recovery Model for the Upper Extremity After an Ischemic Stroke. Neurorehabil Neural Repair. 2015;29(7):614–22.PubMedView ArticleGoogle Scholar
- Kwakkel G, Kollen B, Twisk J. Impact of time on improvement of outcome after stroke. Stroke. 2006;37(9):2348–53.PubMedView ArticleGoogle Scholar
- Kwakkel G, Veerbeek JM, van Wegen EEH, Wolf SL. Constraint-induced movement therapy after stroke. Lancet Neurol. 2015;14(2):224–34.PubMedPubMed CentralView ArticleGoogle Scholar
- Ward NS. Does neuroimaging help to deliver better recovery of movement after stroke? Curr Opin Neurol. 2015;28(4):323–9.PubMedView ArticleGoogle Scholar
- Radlinska B, Ghinani S, Leppert IR, Minuk J, Pike GB, Thiel A. Diffusion tensor imaging, permanent pyramidal tract damage, and outcome in subcortical stroke. Neurology. 2010;75(12):1048–54.PubMedPubMed CentralView ArticleGoogle Scholar
- Puig J, Pedraza S, Blasco G, Daunis-I-Estadella J, Prados F, Remollo S, Prats-Galino A, Soria G, Boada I, Castellanos M, Serena J. Acute damage to the posterior limb of the internal capsule on diffusion tensor tractography as an early imaging predictor of motor outcome after stroke. AJNR Am J Neuroradiol. 2011;32(5):857–63.PubMedView ArticleGoogle Scholar
- Schulz R, Park C-H, Boudrias M-H, Gerloff C, Hummel FC, Ward NS. Assessing the integrity of corticospinal pathways from primary and secondary cortical motor areas after stroke. Stroke. 2012;43(8):2248–51.PubMedPubMed CentralView ArticleGoogle Scholar
- Park C-H, Kou N, Boudrias M-H, Playford ED, Ward NS. Assessing a standardised approach to measuring corticospinal integrity after stroke with DTI. NeuroImage Clin. 2013;2:521–33.PubMedPubMed CentralView ArticleGoogle Scholar
- Kou N, Park C, Seghier ML, Leff AP, Ward NS. Can fully automated detection of corticospinal tract damage be used in stroke patients? Neurology. 2013;80(24):2242–5.PubMedPubMed CentralView ArticleGoogle Scholar
- Price CJ, Seghier ML, Leff AP. Predicting language outcome and recovery after stroke: the PLORAS system. Nat Rev Neurol. 2010;6(4):202–10.PubMedPubMed CentralView ArticleGoogle Scholar
- Hope TMH, Seghier ML, Leff AP, Price CJ. Predicting outcome and recovery after stroke with lesions extracted from MRI images. NeuroImage Clin. 2013;2:424–33.PubMedPubMed CentralView ArticleGoogle Scholar
- Atluri G, Padmanabhan K, Fang G, Steinbach M, Petrella JR, Lim K, Macdonald A, Samatova NF, Doraiswamy PM, Kumar V. Complex biomarker discovery in neuroimaging data: Finding a needle in a haystack. NeuroImage Clin. 2013;3:123–31.PubMedPubMed CentralView ArticleGoogle Scholar
- Ward NS. Getting lost in translation. Curr Opin Neurol. 2008;21(6):625–7.PubMedView ArticleGoogle Scholar
- Saur D, Ronneberger O, Kümmerer D, Mader I, Weiller C, Klöppel S. Early functional magnetic resonance imaging activations predict language outcome after stroke. Brain A J Neurol. 2010;133(Pt 4):1252–64.View ArticleGoogle Scholar
- Stinear CM, Barber PA, Smale PR, Coxon JP, Fleming MK, Byblow WD. Functional potential in chronic stroke patients depends on corticospinal tract integrity. Brain. 2007;130(Pt 1):170–80.PubMedGoogle Scholar
- Riley JD, Le V, Der-Yeghiaian L, See J, Newton JM, Ward NS, Cramer SC. Anatomy of stroke injury predicts gains from therapy. Stroke. 2011;42(2):421–6.PubMedPubMed CentralView ArticleGoogle Scholar
- Cramer SC, Parrish TB, Levy RM, Stebbins GT, Ruland SD, Lowry DW, Trouard TP, Squire SW, Weinand ME, Savage CR, Wilkinson SB, Juranek J, Leu SYY, Himes DM. Predicting functional gains in a stroke trial. Stroke. 2007;38(7):2108–14.Google Scholar
- Gerloff C, Hallett M. Big news from small world networks after stroke. Brain A J Neurol. 2010;133(Pt 4):952–5.View ArticleGoogle Scholar
- Friston KJ, Harrison L, Penny W. Dynamic causal modelling. Neuroimage. 2003;19(4):1273–302.PubMedView ArticleGoogle Scholar
- Chen C-C, Kiebel SJ, Kilner JM, Ward NS, Stephan KE, Wang W-J, Friston KJ. A dynamic causal model for evoked and induced responses. Neuroimage. 2012;59(1):340–8.PubMedPubMed CentralView ArticleGoogle Scholar
- Friston KJ. The labile brain. I. Neuronal transients and nonlinear coupling. Philos Trans R Soc Lond B Biol Sci. 2000;355(1394):215–36.PubMedPubMed CentralView ArticleGoogle Scholar
- Falcon MI, Riley JD, Jirsa V, McIntosh AR, Shereen AD, Chen EE, Solodkin A. The Virtual Brain: Modeling Biological Correlates of Recovery after Chronic Stroke. Front Neurol. 2015;6:228.PubMedPubMed CentralView ArticleGoogle Scholar
- Yamawaki N, Stanford IM, Hall SD, Woodhall GL. Pharmacologically induced and stimulus evoked rhythmic neuronal oscillatory activity in the primary motor cortex in vitro. Neuroscience. 2008;151(2):386–95.PubMedView ArticleGoogle Scholar
- Traub RD, Bibbig A, LeBeau FEN, Buhl EH, Whittington MA. Cellular mechanisms of neuronal population oscillations in the hippocampus in vitro. Annu Rev Neurosci. 2004;27:247–78.PubMedView ArticleGoogle Scholar
- Moran RJ, Stephan KE, Kiebel SJ, Rombach N, O’Connor WT, Murphy KJ, Reilly RB, Friston KJ. Bayesian estimation of synaptic physiology from the spectral responses of neural masses. Neuroimage. 2008;42(1):272–84.PubMedPubMed CentralView ArticleGoogle Scholar
- Moran RJ, Jung F, Kumagai T, Endepols H, Graf R, Dolan RJ, Friston KJ, Stephan KE, Tittgemeyer M. Dynamic causal models and physiological inference: a validation study using isoflurane anaesthesia in rodents. PLoS One. 2011;6(8):e22790.PubMedPubMed CentralView ArticleGoogle Scholar
- Muthukumaraswamy SD, Carhart-Harris RL, Moran RJ, Brookes MJ, Williams TM, Errtizoe D, Sessa B, Papadopoulos A, Bolstridge MK, Singh D, Feilding A, Friston KJ, Nutt DJ. Broadband cortical desynchronization underlies the human psychedelic state. J Neurosci Off J Soc Neurosci. 2013;33(38):15171–83.Google Scholar
- Takiyama K, Okada M. Recovery in stroke rehabilitation through the rotation of preferred directions induced by bimanual movements: a computational study. PLoS One. 2012;7(5):e37594.PubMedPubMed CentralView ArticleGoogle Scholar
- Patten C, Condliffe EG, Dairaghi CA, Lum PS. Concurrent neuromechanical and functional gains following upper-extremity power training post-stroke. J Neuroeng Rehabil. 2013;10:1.PubMedPubMed CentralView ArticleGoogle Scholar
- Page SJ, Gater DR, Bach-Y-Rita P. Reconsidering the motor recovery plateau in stroke rehabilitation. Arch Phys Med Rehabil. 2004;85(8):1377–81.PubMedView ArticleGoogle Scholar
- Brouwer BJ. Hand function and motor cortical output poststroke: are they related? Arch Phys Med Rehabil. 2006;87(5):627–34.PubMedView ArticleGoogle Scholar
- Ward NS, Newton JM, Swayne OB, Lee L, Frackowiak RS, Thompson AJ, Greenwood RJ, Rothwell JC. The relationship between brain activity and peak grip force is modulated by corticospinal system integrity after subcortical stroke. Eur J Neurosci. 2007;25(6):1865–73.PubMedPubMed CentralView ArticleGoogle Scholar
- Noskin O, Krakauer JW, Lazar RM, Festa JR, Handy C, O’Brien KA, Marshall RS. Ipsilateral motor dysfunction from unilateral stroke: implications for the functional neuroanatomy of hemiparesis. J Neurol Neurosurg Psychiatry. 2008;79(4):401–6.PubMedView ArticleGoogle Scholar
- Hidaka Y, Han CE, Wolf SL, Winstein CJ, Schweighofer N. Use it and improve it or lose it: interactions between arm function and use in humans post-stroke. PLoS Comput Biol. 2012;8(2):e1002343.PubMedPubMed CentralView ArticleGoogle Scholar
- Reinkensmeyer DJ. “How to Retrain Movement after Neurologic Injury: A Computational Rationale for Incorporating Robot (or Therapist) Assistance”. Proc IEEE Eng Med Biol Soc Meet. 2003;2:1479–1482.View ArticleGoogle Scholar
- Tee KP, Franklin DW, Kawato M, Milner TE, Burdet E. “Concurrent adaptation of force and impedance in the redundant muscle system. Biol Cybern. 2010;102:31–44.PubMedView ArticleGoogle Scholar
- Emken JL, Benitez R, Reinkensmeyer DJ. Human-robot cooperative movement training: learning a novel sensory motor transformation during walking with robotic assistance-as-needed. J Neuroeng Rehabil. 2007;4:8.PubMedPubMed CentralView ArticleGoogle Scholar
- Jarrassé N, Charalambous T, Burdet E. A framework to describe, analyze and generate interactive motor behaviors. PLoS One. 2012;7(11):e49945.PubMedPubMed CentralView ArticleGoogle Scholar
- Hu XL, Tong K-YY, Song R, Zheng XJ, Leung WWF. A comparison between electromyography-driven robot and passive motion device on wrist rehabilitation for chronic stroke. Neurorehabil Neural Repair. 2009;23(8):837–46.PubMedView ArticleGoogle Scholar
- Grefkes C, Fink GR. Connectivity-based approaches in stroke and recovery of function. Lancet Neurol. 2014;13(2):206–16.PubMedView ArticleGoogle Scholar
- Bland MD, Beebe JA, Hardwick DD, Lang CE. Restricted active range of motion at the elbow, forearm, wrist, or fingers decreases hand function. J Hand Ther. 2008;21(3):268–74.PubMedView ArticleGoogle Scholar
- C. E. Lang, M. D. Bland, R. R. Bailey, S. Y. Schaefer, and R. L. Birkenmeier. Assessment of upper extremity impairment, function, and activity after stroke: foundations for clinical decision making. J. Hand Ther. 26 (2): 104–14. quiz 115.Google Scholar
- Nahmani M, Turrigiano GG. Adult cortical plasticity following injury: Recapitulation of critical period mechanisms? Neuroscience. 2014;283:4–16.PubMedView ArticleGoogle Scholar
- Reis J, Schambra HM, Cohen LG, Buch ER, Fritsch B, Zarahn E, Celnik PA, Krakauer JW. Noninvasive cortical stimulation enhances motor skill acquisition over multiple days through an effect on consolidation. Proc Natl Acad Sci U S A. 2009;106(5):1590–5.PubMedPubMed CentralView ArticleGoogle Scholar
- Hummel F, Celnik P, Giraux P, Floel A, Wu W-H, Gerloff C, Cohen LG. Effects of non-invasive cortical stimulation on skilled motor function in chronic stroke. Brain. 2005;128(Pt 3):490–9.PubMedView ArticleGoogle Scholar
- Reinkensmeyer DJ, Wolbrecht ET, Chan V, Chou C, Cramer SC, Bobrow JE. Comparison of 3D, assist-as-needed robotic arm/hand movement training provided with Pneu-WREX to conventional table top therapy following chronic stroke. Am J Phys Med Rehabil. 2012;91(11 Suppl 3):S232–41.PubMedPubMed CentralView ArticleGoogle Scholar
- J. Rowe, N. Friedman, V. Chan, S. Cramer, M. Bachman, D. J. Reinkensmeyer. The Variable Relationship between Arm and Hand Use: A Rationale for Using Finger Magnetometry to Complement Wrist Accelerometry When Measuring Daily Use of the Upper Extremity. Proc. 36th Annu. Int. IEEE EMBS Conf. 2014.Google Scholar