Skip to content

Advertisement

  • Research
  • Open Access

Development of a 3D, networked multi-user virtual reality environment for home therapy after stroke

  • 1Email author,
  • 2,
  • 1,
  • 1,
  • 1 and
  • 3, 4
Journal of NeuroEngineering and Rehabilitation201815:88

https://doi.org/10.1186/s12984-018-0429-0

  • Received: 24 October 2017
  • Accepted: 18 September 2018
  • Published:

Abstract

Background

Impairment of upper extremity function is a common outcome following stroke, to the detriment of lifestyle and employment opportunities. Yet, access to treatment may be limited due to geographical and transportation constraints, especially for those living in rural areas. While stroke rates are higher in these areas, stroke survivors in these regions of the country have substantially less access to clinical therapy. Home therapy could offer an important alternative to clinical treatment, but the inherent isolation and the monotony of self-directed training can greatly reduce compliance.

Methods

We developed a 3D, networked multi-user Virtual Environment for Rehabilitative Gaming Exercises (VERGE) system for home therapy. Within this environment, stroke survivors can interact with therapists and/or fellow stroke survivors in the same virtual space even though they may be physically remote. Each user’s own movement controls an avatar through kinematic measurements made with a low-cost, Kinect™ device. The system was explicitly designed to train movements important to rehabilitation and to provide real-time feedback of performance to users and clinicians. To obtain user feedback about the system, 15 stroke survivors with chronic upper extremity hemiparesis participated in a multisession pilot evaluation study, consisting of a three-week intervention in a laboratory setting. For each week, the participant performed three one-hour training sessions with one of three modalities: 1) VERGE system, 2) an existing virtual reality environment based on Alice in Wonderland (AWVR), or 3) a home exercise program (HEP).

Results

Over 85% of the subjects found the VERGE system to be an effective means of promoting repetitive practice of arm movement. Arm displacement averaged 350 m for each VERGE training session. Arm displacement was not significantly less when using VERGE than when using AWVR or HEP. Participants were split on preference for VERGE, AWVR or HEP. Importantly, almost all subjects indicated a willingness to perform the training for at least 2–3 days per week at home.

Conclusions

Multi-user VR environments hold promise for home therapy, although the importance of reducing complexity of operation for the user in the VR system must be emphasized. A modified version of the VERGE system is currently being used in a home therapy study.

Keywords

  • Stroke
  • Rehabilitation
  • Virtual reality
  • Serious games
  • Upper extremity

Background

Chronic upper extremity impairment is all too common among the more than 7 million stroke survivors in the U.S. [1]. These impairments have disabling effects on all facets of life, including self-care, employment, and leisure activities. Repetitive practice of movement, such as arm movement, is thought to improve outcomes for stroke survivors [24], but access to the clinic for therapy is often limited by geography or lack of transportation. While almost 50 million Americans live in rural areas, 90% of physical and occupational therapists live in major urban areas [5]. Per capita ratios of therapists to overall population are 50% larger in urban as compared to rural regions of the country [6]. Rates of stroke in these rural areas, however, exceed those of major urban areas [79]. Thus, a large number of stroke survivors have limited access to skilled treatment. Data from 21 states found that only 30% of stroke survivors received outpatient rehabilitation, a much lower percentage than that recommended by clinical practice guidelines [10]. Declines seen following discharge from inpatient rehabilitation are undoubtedly exacerbated by limited access to clinical therapy [11].

Disparity in quality of care has been recognized in the acute treatment of stroke for a number of years. This situation has led to the development of telemedicine to extend expert care to individuals during the initial hours and days following the stroke, advance site-independent treatment, and create models of care in rural areas [1214]. Therapy options after this acute period, however, generally remain limited for stroke survivors in rural areas. Akin to the telemedicine efforts, telerehabilitation treatments have been proposed. However, telerehabilitation interactions are typically limited to off-line monitoring by the therapist [8, 9, 15], phone calls between a therapist and client [16, 17], or videoconferencing [1820]. While systems allowing more direct interaction have been proposed, the hardware cost and complexity limit applicability for home-based therapy [2123]. Hence, the therapist is relegated to the role of observer and the intimacy of a clinical therapy session is lost. Therapy options are substantially restricted, as is the available feedback.

Recently, multiple investigators have been exploring means of improving home-based therapy through the development of systems or serious games which permit multiple, simultaneous users [2430]. These efforts have proposed the inclusion of multiple users as a means to overcome resistance to home-based therapy that may result due to isolation or lack of engagement. Indeed, studies have observed a preference for multi-user vs, single-user therapy when utilizing these systems [26, 29]. However, these systems have largely been limited to control of a one-dimensional or two-dimensional space and both users remain in the same physical location (e.g., side by side). One team of researchers did develop a framework for supporting distant users (such as a therapist in the hospital and a stroke survivor in their home), but game control was limited to one or two dimensions [31, 32].

Here, we describe the development of a fully three-dimensional (3D) virtual reality environment (VRE) for home-based therapy in which multiple, remote users can interact in real time. In this Virtual Environment for Rehabilitative Gaming Exercises (VERGE) system [33], movement of the user is mapped to corresponding movement of an avatar to foster a sense of presence in and engagement with the VRE. The 3D environment encompasses aspects of clinical therapy, such as transport of objects or movement of the hand into specified regions of the upper extremity workspace. Although the importance of 3D movements in VR environments is a topic of debate [34, 35], movements tested in environments with lesser degrees-of-freedom (DOF) are often very limited and dictated by a one DOF robot. These movements differ substantially from the types of movements normally seen in 3D reaching movements [4, 36]. The network architecture of the system allows users to be located remotely from each other, such as a stroke survivor in their home, a therapist in a clinic, or a stroke survivor’s friend or relative living in another city or state. The virtual nature of the environment allows even very limited movements in the physical world to have successful functional outcomes in the virtual world, thereby offering a sense of accomplishment and motivation for successive attempts. Additionally, task difficulty can easily be modified in order to maintain the proper level of challenge, which is important for motor learning in general [37] and rehabilitation in particular [38].

We developed and performed preliminary testing of the VERGE system to gauge user response in comparison to two other therapy modalities that could be used for home therapy: an existing virtual reality system based on the Alice in Wonderland story (AWVR) [39] and a home exercise program (HEP). Fifteen stroke survivors completed three, one-hour therapy sessions per week with each of the three therapy modalities (9 sessions total). We hypothesized that the use of the VERGE system would not decrease the amount of arm movement promoted, in comparison with the AWVR and HEP modalities. We further expected that users’ self-described engagement would be greatest for the VERGE system due to the presence of a partner.

Methods

VERGE System

Architecture

At its core, VERGE consists of a 3D VRE in which avatars interact with virtual objects. To date, we have created two such VREs, one depicting a dining room and the other a kitchen. The scenes were created in Maya (Autodesk Inc., San Rafael, CA) and imported into Unity 3D (Unity 4.5, Unity Technologies, San Francisco, CA), the software platform controlling VERGE. The VREs are rich in detail in order to provide depth cues [40]. Thus, depth can be conveyed without the need for stereovision, such as that provided by head mounted displays (HMDs). We have found that HMDs can be difficult for stroke survivors to use due to the limited field-of-view and, especially, involuntary coupling between neck and arm motion [41, 42]. The latter may lead to complications with moving the arm while keeping the head steady.

The avatars were created from a custom skeleton in Maya (Autodesk Inc., San Rafael, CA), which was rigged to an existing mesh of the “casual young man” 3D model, purchased and modified for our project (Fig. 1). We created the custom skeleton to match the topology of the existing character while corresponding to the skeletal joint naming convention in Unity 3D. The skeleton (and thus avatar) is animated according to joint angle data captured with a Kinect™ I optical tracker (Microsoft Corp., Redmont, WA). The 3D motion data from the Kinect™ are transmitted to the Unity code through UDP to drive the movement of the avatar in the virtual environment.
Fig. 1
Fig. 1

Avatar kinematics. a 3D model of the avatar with underlying custom skeleton as displayed in Maya. b Avatar imported into custom scene in Unity. Visible coordinate frame indicates location of right hip joint. Ellipsoid encompassing hands represents the contact regions for the hand colliders

The VERGE system employs a central server interacting with peripheral client computers, one for each user. The server receives information from the client computers and controls updating of the scene so that the appropriate view of the scene is shown on each client computer through TCP/IP network architecture. We implemented communication between the client computers through custom libraries in C# (Microsoft Visual Studio). We used two multi-user network models: an authoritative server (server performs all physics calculations) for when more than one user could interact with virtual objects simultaneously and a non-authoritative server (local computation of physics) for when only one user at a time could interact with the virtual environment.

Exercises

We have created three exercises (Ball Bump, Food Fight, and Trajectory Trace) employing Unity 3D and C#. These exercise were designed to encourage upper extremity movement, particularly to areas of the workspace that are often difficult to reach, such as those requiring raising the arm and reaching away from the body [43]. For each exercise, VERGE provides a first-person view of the virtual scene, as through the eyes of the avatar, to each user in accordance with previous studies [44]. This helps to establish a sense of presence for the user in the scene. The server displays a third-person view of the VRE.

Ball Bump is played on the table of the dining room VRE created in Maya (Fig. 2a). The goal is to hit a ball back and forth across the table, while avoiding the objects on the table. Contact between the ball and the avatar hand produces a collision that redirects the ball according to the Unity physics engine. Similarly, collisions between the ball and other objects redirect the ball. The ball will fall off the table if a fellow participant misses making contact with it and /or if the user hits the ball in the wrong direction. Pressing a red napkin, located to the side, produces a new ball. Thus, participants are encouraged to reach away from their bodies, especially to contact the napkin or to free the ball when it becomes stuck behind an object. This can be a collaborative exercise, in which the participants try to make as many successful passes as possible before the ball falls off the table, or a competitive game in which each player tries to hit the ball past the other player. Prior studies have shown that different users will have different preferences for competitive or collaborative exercises [30].
Fig. 2
Fig. 2

Three exercises developed for the VERGE system. First-person view of the user (client) is shown for each exercise. The inset image shows the corresponding third-person server view. a Ball Bump. Users pass a ball back and forth across the Table. b Food Fight. Players pick up food items and throw them at each other. c Trajectory Trace. One player draws a trajectory in space; another player must retrace the trajectory to erase it

The Food Fight game takes place in the custom kitchen VRE created in Maya. Participants grab different food items and throw them at other avatars (Fig. 2b). The user “grasps” an object by placing the avatar’s hand in close proximity and clicking a button on a wireless optical pen mouse (2.4GHz Wireless Optical Pen Mouse Adjustable 500/1000DPI, Docooler) with either hand (stroke survivors typically operate the pen mouse with the ipsilesional hand). The user releases the object by operating another button on the pen mouse; the object’s trajectory is determined by the object’s velocity vector at the time of release. Once all of the food items have been thrown (e.g., cake, eggs, and fruit), the user can reset the food items by clicking on the reset button to continue play. We integrated mesh deformations and special effects for food items colliding with other objects. Eggs splatter and appear as yolks, the cake morphs into a pile of crumbs, and the pears and apples deform into broken fruit. Scenes are updated by the central server in authoritative mode.

In the Trajectory Trace exercise (Fig. 2c) one participant draws a 3D trajectory in the air. This trajectory is then passed to another participant who attempts to erase it by retracing. The state of the game (Draw, Claim, Trace, or Reset), as well as the initiation and termination of drawing the curve, is controlled by touching a button (located on the avatar’s chest) with the less affected hand. The trajectory is anchored to the avatar, such that the user must reach with the arm rather than using the trunk to move the hand to the trajectory, similar to a previous study [45]. The partner (such as a therapist) can specify to which part of the workspace the other player (e.g., a stroke survivor) should practice reaching, by drawing the trajectory in that region. To help with depth perception, a translucent 3D cube outlines the boundaries of the drawn 3D shape. Since the trajectory must be retraced in the same direction in which it was created, we display a yellow sphere to indicate the current starting point. As the user successfully traces the trajectory and it disappears, the 3D depth cube adjusts to match the volume of the remaining trajectory in real time.

Pilot Study

A pilot study was performed to examine the feasibility of the system for use by stroke survivors. Especially, we wanted to test the amount of movement of the impaired arm, particularly to certain regions of the upper extremity workspace. Specifically, we wanted to determine whether employment of the VERGE system reduced arm movement in comparison with other potential home treatment options, namely the AWVR and HEP. We also wanted to compare user engagement across the three modalities.

Participants

Fifteen stroke survivors (10 male/ 5 female) in the chronic stage of recovery participated in this study. Subjects were at least 2 years post-stroke (mean of 17.4 years) and ranged in age from 33 to 81 years. Subjects had upper extremity hemiparesis, rated as Stage 3 - Stage 5 on the Stage of Arm subsection of the Chedoke-McMaster Stroke Assessment Scale [46] by a trained therapist. Subjects had no known orthopedic disease, significant visual deficits, contracture or pain (self-reported pain less than 6 on a 10-point scale) in the arm that would have hampered performing the experiments. Northwestern University’s Institutional Review Board (Chicago, IL) approved the study design and each participant signed an informed consent before study enrollment.

Intervention

Each participant took part in a three-week intervention study consisting of 9 one-hour training sessions in a laboratory setting over 3 weeks. During each week, the participant was involved in one of three therapy modalities 1) VERGE, 2) HEP or 3) AWVR. The order of the therapy was randomized for each participant, and all participants took part in all three therapies.

The VERGE therapy utilized the three different exercises previously described. Ball Bump was always the first exercise introduced to the participant as it was deemed the simplest to understand, and thus provided a straightforward means for learning to control the avatar in the VRE. Trajectory Trace and Food Fight were then played, in that order. All exercises for the VERGE therapy were performed with the arm unsupported. Participants performed each exercise for 15 min, with a 5 min resting period between games. For the purposes of this study, the other user was always a member of the study team with experience performing the virtual exercises, in a situation akin to the expert user employed in an aforementioned study [29]. This individual was located in a different room within the same building to simulate home use.

The HEP therapy consisted of pre-defined sets of seated, self-paced arm-hand exercises derived from standard care. These 16 exercises were presented to the participant in the form of a printed handout (see Additional file 1). The HEP consisted of a generalized list of tasks for the specific purposes of this research study, and not one individualized to the needs of a particular patient. In accordance with the other modalities, we instructed participants to work through the handout in three blocks of 15 min of activity followed by 5 min of rest for each training session. Participants performed all tasks while seated at a table. Roughly 30% of the exercises involved arm support (provided by the tabletop) while 70% were unsupported. To limit the potential for participants to perform extra HEP at home, no copies of the HEP protocol were permitted to leave the research facility.

The AWVR is a rich VRE on which we have trained stroke survivors in the past [39]. The environment draws the user into the March Hare’s tea party where the participant is guided to perform a number of tasks to spur repetitive practice of movements. We chose three of the available exercises, based on their appropriateness for encouraging arm movement. One exercise, Tea Stir, entailed picking up a spoon from a jar and then reaching forward to stir tea in a teacup; the arm remained suspended, unsupported until the spoon “melted” and disappeared (Fig. 3a). This exercise involved crossing the midline of the body and extending the arm laterally. For the Crabby Cookies exercise, the participant reached out to a plate of virtual cookies. Once a cookie was touched, it transformed into a crab that scurried across the table and had to be “caught” by touching it (Fig. 3b). Thus, the Crabby Cookies exercise required multiple reaches away from the body. The third exercise, Bottomless Sherry, required the participant to reach for a glass of sherry, raise it to take a sip, and then set it back on the table to be refilled (Fig. 3c). We instructed participants to place the glass in different locations on the surface of the table to encourage reaching to her range of motion limits. All exercises were performed with the arm unsupported. Participants performed each exercise for 15 min, followed by a 5 min break, over the one-hour session. As the complexity was similar across exercises, order of play was random and often chosen by the participant.
Fig. 3
Fig. 3

Three AWVR exercises used for this study. First-person user view is shown. a Tea Stir. Spoons (foreground) melt when used to stir the hot tea. b Crabby Cookies. Cookies morph into crabs that run around the table and must be “captured”. c Bottomless Sherry. Sherry glass must be repeatedly lifted to “drink” and set down on the table to refill

During the first therapy session for each therapy modality, study staff explained the exercises and demonstrated performance. After that, study staff were available throughout each session to answer questions and monitor for safety, but each participant was encouraged to work independently, as if at home. As noted for VERGE, the participant worked together with another member of the study staff, but that individual was located in a separate room.
Fig. 4
Fig. 4

Stroke survivor training while wearing Xsens. Participant is trying to erase the displayed trajectory in the Trajectory Trace exercise in VERGE. While control of the avatar is provided through the Kinect™, the subject is wearing an Xsens 3D motion tracker system vest (MVN, Xsens, Culver City, CA) to provide continuous measurement of the hand and shoulder displacement for experimental analysis

Outcome measures

As we were especially interested in user response to VERGE, we administered a questionnaire (VERGE Survey) employing a 5-point Likert scale [47, 48] at the end of the VERGE training week (see Additional file 2). This questionnaire addressed the different exercises and issues specific to VERGE. Participants completed a different questionnaire (Weekly Survey) at the end of each therapy week (including VERGE) to measure the level of engagement in and the perceived potential for the therapy (See Additional file 3). We administered a final questionnaire (Final Survey), directly comparing the three therapies, at the end of the study (See Additional file 4). Additionally, we captured participant kinematics throughout the third session for each therapy modality with the Xsens 3D motion tracker system (MVN, Xsens, Culver City, CA), (Fig. 4). Participants donned the Xsens vest, containing eight inertial measurement units (IMUs) and a headband containing one IMU (per upper extremity configuration). The Xsens system continuously recorded upper extremity movement from the IMU data during all of the exercises.

Analysis

Descriptive statistics were utilized to compare responses from the VERGE Survey. For the Weekly Surveys, we used the nonparametric Friedman test to compare participant responses, as measured by the Likert scores, across the treatment modalities. Hand and shoulder displacements were computed from the Xsens data by using the biomechanical toolkit Mokka [49]. Total shoulder movement was subtracted from total hand movement to account for arm displacement resulting purely from trunk translation. We performed non-inferiority tests to examine whether total arm displacement during VERGE was inferior to arm displacement during the other two modalities. The δ was set equal to 10% of the larger of the hand displacement means for AWVR and HEP. Furthermore, we examined which parts of the workspace were accessed. For example, Reach Distance was calculated as the relative distance, normalized by arm length, of the hand away from the shoulder. Thus, a Reach Distance = 0 indicated that the hand was coincident with the shoulder, while a Reach Distance = 1 indicated full arm extension. We defined Hand Elevation as the vertical location of the hand with respect to the shoulder. A value of 0 represented the hand at shoulder height, while a value of 1 indicated that the hand was located at its lowest possible position with respect to the shoulder (i.e., hand at the side with elbow fully extended). We performed non-inferiority tests to examine whether time spent with a Reach Distance ≥ 0.7 and whether the time spent with Hand Elevation ≤ 0.4 were inferior for VERGE than for the other two modalities. The δ was set to 4.5 min, equal to 10% of the total training time. We also created histograms to show the amount of time the hand was positioned at different bins of Reach Distance and Hand Elevation.

Results

Questionnaires

User feedback gathered for the VERGE Survey was generally positive, with 13 of 15 participants indicating that the therapy was Very (n = 8) or Extremely (n = 5) productive and 14 of 15 participants indicating that they were Satisfied (n = 2) or Very Satisfied (n = 12) with the amount of arm use during the therapy session. Additionally, participants largely enjoyed having a partner as 13 of the 15 participants Very Much (n = 7) or Extremely (n = 6) enjoyed playing with a virtual partner and 14 of 15 participants Agreed (7) or Strongly agreed (7) that training with another virtual partner in the environment increased motivation. Response to the look of the exercises was generally positive as 11 of 15 participants Very Much (n = 6) or Extremely (n = 5) liked the 3D graphics of the system. Importantly, 12 of 15 participants Agreed (n = 5) or Strongly Agreed (n = 7) that the VERGE system had great potential as home-based rehabilitation.

Results of the Weekly Survey, administered at the end of each training modality, are shown in Table 1. Few statistically significant differences were apparent among the three treatment modalities for any of the responses. Satisfaction with time spent in training was significantly less for VERGE than for the other modalities; however, all three modalities showed a mean response above 4, which represented Satisfied on the survey. Responses were generally quite positive, with the vast majority of responses having a mean value of 4.0 or greater (5-point Likert scale with 3 indicating a neutral response). Importantly, the majority of subjects indicated that if the equipment were available they would Definitely (n = 10) or Probably (n = 4) continue at least one of the therapy modalities at home. Overall, they indicated a willingness to perform the therapy for 2–3 or more days per week (n = 15).
Table 1

Results for Weekly Survey administered at the end of week for each modality

Survey Question

VERGE

AWVR

HEP

Friedman

Importance of exercise speed

4.6 (0.6)

4.3 (1.1)

4.6 (0.7)

0.733

Importance of personal progress

4.9 (0.3)

4.5 (0.7)*

4.9 (0.4)

0.038

Importance of performance of activities of daily living

4.6 (0.6)

4.6 (0.8)

4.9 (0.4)

0.522

Importance of ability to perform new tasks

4.5 (1.0)

4.6 (0.7)

4.9 (0.3)

0.264

Importance of greater arm/hand movement

4.8 (0.4)

4.9 (0.5)

4.9 (0.4)

0.717

Level of interest/stimulation from therapy

4.0 (1.0)

4.2 (0.7)

4.1 (1.0)

0.889

Effectiveness/helpfulness for arm/hand

4.2 (0.8)

4.4 (0.6)

4.2 (1.0)

0.368

Satisfaction with ease of use

4.3 (1.0)

4.5 (0.8)

4.4 (1.0)

0.661

Satisfaction with your attention to exercises

4.6 (0.8)

4.3 (0.8)

4.7 (0.6)

0.206

Satisfaction with your desire to complete training

4.8 (0.8)

4.9 (0.4)

4.7 (0.5)

0.311

Satisfaction with progress

4.2 (0.8)

4.3 (0.7)

4.6 (0.6)

0.061

Satisfaction with amount of arm use

4.7 (0.8)

4.7 (0.5)

4.7 (0.5)

1.0

Satisfaction with time spent in training

4.1 (0.8)*

4.6 (0.5)

4.6 (0.5)

0.032

Progress experienced in training

3.7 (1.1)

3.9 (0.9)

3.5 (1.1)

0.565

Amount of arm movement compared to prior treatment

4.1 (0.8)

4.5 (0.6)

4.3 (0.8)

0.439

Continued home use

3.9 (1.4)

4.3 (0.8)

4.0 (1.4)

0.304

Expected frequency of home exercises

3.2 (1.0)

3.3 (0.7)

3.2 (1.2)

0.595

Rehabilitation potential

4.1 (1.1)

4.1 (0.9)

4.4 (0.7)

0.206

Likert scale from 1 to 5 employed. Higher numbers denote more positive responses. Mean (SD)

* indicates significance at the level indicated by the Friedman test statistic

The questionnaire directly comparing the training modalities (Final Survey) revealed a variety of opinions (Table 2). Participants were fairly split about which of the modalities they found the most engaging (each modality received 5 selections) and which they preferred (5 each selected HEP and AWVR and 4 selected VERGE). However, we observed strong trends favoring HEP regarding ease of understanding (two-thirds of participants selected HEP) and which therapy users would most likely continue in the home (9 of 15 participants selected HEP).
Table 2

Results of Final Survey comparing the three training paradigms

Characteristic

VERGE

AWVR

HEP

Most engaging

5

5

5

Greatest desire to complete sessions

3

6

6

Moved arm the most

4

4

7

Easiest to understand

1

3

10

Most effective

1

5

8

Preferred form of therapy

4

5

5

Most likely to continue at home

3

3

9

Values listed reflect number of subjects choosing each modality

Kinematics

Each therapy modality promoted considerable arm movement. Total arm displacement averaged: 354 m for VERGE, 503 m for HEP, and 229 m for AWVR (Fig. 5). Non-inferiority testing showed that arm displacement produced by participants during the VERGE training was not significantly inferior to HEP or AWVR (Fig. 6a). All three modalities also encouraged extended arm postures and elevated hand positions. Time spent with Reach Distance ≥ 0.7 for VERGE was not inferior to that for the other two modalities (Fig. 6b). Additionally, the largest histogram values occurred for postures at which the hand was extended 70% or more of full-arm length away from the shoulder (Fig. 7a). The VERGE system also promoted elevating the hand at least as much as other modalities. Time spent with Hand Elevation ≤ 0.4 was not inferior for VERGE as compared to AWVR and HEP (Fig. 6c). The histograms revealed that during the VERGE session, subjects spent 18.2% (± 14.3%) of the time with the hand at or within 40% of arm length below shoulder height (Fig. 7b), while they spent 10.6% (± 0.9%,) with the AWVR and 16.5% (± 12.3%) during the HEP.
Fig. 5
Fig. 5

Average total arm displacement all subjects for each of the three training modalities. Movement tracked across third training session for each modality. Error bar represents one standard deviation

Fig. 6
Fig. 6

Non-inferiority tests. a Total arm displacement. The δ was set equal to 10% of HEP mean. b Time spent (in minutes) with Reach Distance greater than or equal to 70% of arm length. c Time spent (in minutes) with Hand Elevation within 40% of arm length with respect to the shoulder. The δ was set equal to 10% of full training time (45 min) for b and c. VERGE not inferior to HEP or AWVR for any of these measures

Fig. 7
Fig. 7

Histograms, averaged across all subjects, depicting time the hand of the impaired limb spent in different regions of the workspace. a Reach Distance. Distance of the hand from the shoulder, represented as a fraction of arm length (0: hand coincident with shoulder; 1: arm fully extended). b Hand Elevation Distance. Vertical location of the hand with respect to the shoulder, represented as a fraction of arm length (0, hand elevation equal to shoulder elevation; 1: hand elevation full arm length below shoulder elevation; negative values indicate hand elevation above shoulder elevation). Blue: VERGE, green: AWVR, orange: HEP

Discussion

VERGE implementation

We developed a 3D, networked VR system allowing users, physically remote from each other, to interact within a virtual environment. Each user controls an avatar in real time by movement of corresponding body segments. These avatars can manipulate virtual objects located within the environment; multiple avatars can even manipulate the same object, such as a ball hit back and forth. Each user needs only have a computer, wireless mouse, and a Kinect™ device. No special software is required for the user, only an executable version of our code and the Kinect SDK.

This VERGE system was successfully tested by 15 stroke survivors with chronic hemiparesis in the upper extremity. User response was generally positive, with 85% of the participants expressing satisfaction with the utility of the therapy and 93% indicating satisfaction with the amount of arm movement induced. Indeed, participants moved their hands an average of 350 m (after subtracting shoulder translation) during each session. This far exceeds the amount of hand displacement produced by the 54 movements observed during a typical occupational therapy session [50]. In accordance with previous multi-user training studies [26, 29, 30], the vast majority (14 of 15) participants indicated that they liked having a partner for therapy, despite not being in visual contact with this person.

Comparison with other potential training modalities

Overall, all three therapy options encouraged considerable movement of the hand in space. Non-inferiority testing confirmed that use of the VERGE system did not result in significantly less displacement of the hand than that recorded using the more established AWVR or HEP modalities. As relatively few studies have quantified arm movement during therapy outside of a robotic device, these values provide an important target for therapy. Importantly, all three training modalities encouraged movement away from the body. Arm movements to areas of the workspace which require elbow extension can be challenging for stroke survivors, especially when the arm is unsupported [51, 52], as was the case for VERGE, AWVR, and the majority of HEP. The differences observed in the amount of arm movement between modalities could be a result of confounding factors in exercise design. Specifically, exercises in VERGE were designed to include movements out of synergy and in a large free 3-D space. Although the importance of 3D movements in therapy is a topic of debate [34, 35], many tasks require non-planar movements. VERGE allows practice of such task-based motions. AWVR also included movements out of synergy but the workspace was much more limited in size. HEP included many exercises with proximal arm stabilization, these movements were simpler in that they did not require multiple joint coordination or trunk stabilization.

During the training sessions, participants spent the most time with their arms extended at least 70% of full range. Participants also spent a considerable portion of time with the hand raised in an upper level of the workspace (within 40% of arm length of the shoulder elevation). With VERGE, for example, participants spent almost 20% of the session with their hand in this region of the workspace despite not having arm support.

Users, however, indicated differences in experience across the treatment modalities. While participants rated the modalities similarly in the weekly questionnaires, they expressed a preference for the HEP in certain areas in the comparative questionnaire, including as the most effective therapy and the treatment they would most likely continue in the home. Some of the appeal is undoubtedly attributable to the ease of use. Two-thirds of participants chose HEP as the easiest to use. This needs to be addressed in VERGE, as we describe in the following section.

Limitations and lessons learned

The study identified limitations with VERGE that need to be addressed for improved acceptance and utility. For example, the Trajectory Trace exercise placed a significant cognitive demand on users. They were required to actively cycle through a sequence of discrete states for each round (Draw, Claim, Trace, Reset) while coordinating with another player (i.e., one player would draw a trajectory while another would claim and trace it). It was sometimes difficult to determine the current state and to remember which came next. Thus, while almost equal numbers of subjects listed Ball Bump or Food Fight as their favorite VERGE exercise, none listed Trajectory Trace. Despite similar amounts of time spent on each VERGE exercise, hand displacement during Trajectory Trace was less than 50% of the amount seen during the Ball Bump exercise. Partially attributable to this, over 70% of participants chose HEP as the easiest to understand (only one subject picked VERGE). Clearly, reducing complexity of operation for the user of a therapy paradigm is of paramount importance. We have subsequently modified Trajectory Trace to include a visual display indicating the state flow and current state.

Due to the largely collaborative nature of the tasks in VERGE, they included limited quantitative performance measures for the users. Participants stressed the need for objective feedback. As noticeable functional changes may evolve slowly, quantitative assessment of game performance, which can show gains on a much shorter timescale, may provide the motivation needed in the short-term to enable reaching functional milestones. We have subsequently added scoring for each of the exercises. In some cases, both competitive and collaborative scoring is available.

There were limitations with the pilot study as well. While one of the potential benefits of the VERGE system is the inclusion of other players, we did not directly examine preferences for individual vs. partnered training, as previous studies have done [26, 27, 29, 30]. In our study, the three training modalities were quite different from each other. The HEP and AWVR represented existing modalities with potential for use in the home. Factors such as ease of use, the engaging nature of the virtual task, or scoring undoubtedly influenced preferences for the chosen training modality. It should be noted that a large majority of participants expressed enthusiasm for playing with a partner when using VERGE and indicated that they felt that the presence of another user increased motivation. Enthusiasm for multiple players may have been even greater if a friend or relative had served as the playing partner, as was the case in a previous study [26].

Our relatively small sample of participants displayed considerable motivation in repeatedly coming to the laboratory in the hospital for the study and maintaining study adherence. The enthusiasm for therapy may not be as great in the general population. Additionally, only three training sessions were performed with each modality. User responses may have been different after more sessions.

User response may also have been impacted by the fact that this pilot study was performed in the laboratory rather than the home. Coming to the hospital, interacting with people, and receiving compensation may have elevated interest, particularly for the HEP. Compliance rates for conventional home therapy exercise programs have been mixed [5356].

Conclusions

This represents one of the first tests of stroke survivors interacting with a remote user in a 3D virtual environment for therapy. The VERGE system can be directly utilized for home-based therapy with a family member or friend in their home or a therapist in the clinic. The low cost and minimal requirements make it practical for the clinic or home. Most participants expressed satisfaction with the system and enthusiasm for the virtual partner. However, they did stress the importance of ease of use and feedback of performance. Their responses highlighted the need for technology to be sufficiently flexible to accommodate the different goals and preferences of individual users.

Importantly, participants indicated a strong interest in home therapy. Over 66% responded that they would Definitely be willing to continue therapy in the home and 100% responded that they would perform the training at least 2–3 times per week. Two-thirds of participants indicated that they would be willing to perform home-based training 6–7 times per week. While limitations must be addressed, multi-user virtual reality environments hold promise for maintaining engagement in therapy and providing feedback of performance for home users. We are currently undertaking a home therapy study with the VERGE system.

Abbreviations

ANOVA: 

Analysis of variance

AWVR: 

Alice in Wonderland virtual reality

CMSA: 

Chedoke-McMaster stroke assessment

HEP: 

Home Exercise Program

ROM: 

Range of motion

SDK: 

Software development kit

TCP: 

Transmission Control Protocol

UDP: 

User Datagram Protocol

VERGE: 

Virtual Environment for Rehabilitative Gaming Exercises

VR: 

Virtual reality

Declarations

Acknowledgements

The authors would like to thank Jessica Nguyen for her valuable assistance with data analysis and Ning Yuan for his appreciated role with data collection.

Funding

Funded by the US Department of Health and Human Services, National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR), Grant #H133E070013.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

KMT: Research project organization and execution, data acquisition, interpretation, statistical analysis and manuscript writing, review and critique. DT and NS: Design and implementation of the MUVR environment, manuscript review and critique. AJB: Research project execution, data acquisition and analysis, manuscript review and critique. KNT: Design and initial training of the HEP, manuscript review and critique. DGK: Research project conception, design, data review and critique, manuscript writing review and critique. All authors approved the final manuscript.

Ethics approval and consent to participate

The experimental procedure was approved by Northwestern University’s Institutional Review Board (IRB#: STU00200533) and all participants signed informed consent before study enrollment.

Consent for publication

All pictured individuals gave written consent for photographs and publication.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Shirley Ryan AbilityLab, Arms + Hands Lab, Chicago, IL, USA
(2)
School of Design, University of Illinois at Chicago (UIC), Chicago, IL, USA
(3)
UNC/NC State Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
(4)
Closed-Loop Engineering for Advanced Rehabilitation Research Core, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

References

  1. Benjamin EJ, Blaha MJ, Chiuve SE, Cushman M, Das SR, Deo R, de Ferranti SD, Floyd J, Fornage M, Gillespie C, et al. Heart Disease and Stroke Statistics-2017 Update: A Report From the American Heart Association. Circulation. 2017;135:e146–603.View ArticleGoogle Scholar
  2. Kwakkel G. Impact of intensity of practice after stroke: issues for consideration. Disabil Rehabil. 2006;28:823–30.View ArticleGoogle Scholar
  3. Kleim JA, Jones TA. Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage. J Speech Lang Hear Res. 2008;51:S225–39.View ArticleGoogle Scholar
  4. Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, Deruyter F, Eng JJ, Fisher B, Harvey RL, et al. Guidelines for Adult Stroke Rehabilitation and Recovery: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. 2016;47:e98–e169.View ArticleGoogle Scholar
  5. Professions BoH: Rural health profession facts: supply and distribution of health professions in rural america. (Administration HRaS ed. Rockville, MD; 1992.Google Scholar
  6. Analysis NCfHW: Distribution of U.S. health care providers residing in rural and urban areas. US Department of Health and Human Services.Google Scholar
  7. Liao Y, Greenlund KJ, Croft JB, Keenan NL, Giles WH. Factors explaining excess stroke prevalence in the US Stroke Belt. Stroke. 2009;40:3336–41.View ArticleGoogle Scholar
  8. Kairy D, Veras M, Archambault P, Hernandez A, Higgins J, Levin MF, Poissant L, Raz A, Kaizer F. Maximizing post-stroke upper limb rehabilitation using a novel telerehabilitation interactive virtual reality system in the patient's home: study protocol of a randomized clinical trial. Contemp Clin Trials. 2016;47:49–53.View ArticleGoogle Scholar
  9. Wolf SL, Sahu K, Bay RC, Buchanan S, Reiss A, Linder S, Rosenfeldt A, Alberts J. The HAAPI (Home Arm Assistance Progression Initiative) Trial: A Novel Robotics Delivery Approach in Stroke Rehabilitation. Neurorehabil Neural Repair. 2015;29:958–68.View ArticleGoogle Scholar
  10. CfDCa P. Outpatient rehabilitation among stroke survivors: 21 states and the District of Columbia, 2005. Morb Mortal Wkly Rep. 2007;56:504–7.Google Scholar
  11. Hopman WM, Verner J. Quality of life during and after inpatient stroke rehabilitation. Stroke. 2003;34:801–5.View ArticleGoogle Scholar
  12. Meyer BC, Raman R, Hemmen T, Obler R, Zivin JA, Rao R, Thomas RG, Lyden PD. Efficacy of site-independent telemedicine in the STRokE DOC trial: a randomised, blinded, prospective study. Lancet Neurol. 2008;7:787–95.View ArticleGoogle Scholar
  13. Morales-Vidal S, Ruland S. Telemedicine in stroke care and rehabilitation. Top Stroke Rehabil. 2013;20:101–7.View ArticleGoogle Scholar
  14. Muller-Barna P, Hubert GJ, Boy S, Bogdahn U, Wiedmann S, Heuschmann PU, Audebert HJ. TeleStroke units serving as a model of care in rural areas: 10-year experience of the TeleMedical project for integrative stroke care. Stroke. 2014;45:2739–44.View ArticleGoogle Scholar
  15. Housley SN, Garlow AR, Ducote K, Howard A, Thomas T, Wu D, Richards K, Butler AJ. Increasing Access to Cost Effective Home-Based Rehabilitation for Rural Veteran Stroke Survivors. Austin J Cerebrovasc Dis Stroke. 2016;3:1–11.PubMedPubMed CentralGoogle Scholar
  16. Hutchison AJ, Breckon JD. A review of telephone coaching services for people with long-term conditions. J Telemed Telecare. 2011;17:451–8.View ArticleGoogle Scholar
  17. Chumbler NR, Quigley P, Li X, Morey M, Rose D, Sanford J, Griffiths P, Hoenig H. Effects of telerehabilitation on physical function and disability for stroke patients: a randomized, controlled trial. Stroke. 2012;43:2168–74.View ArticleGoogle Scholar
  18. Staszuk A, Wiatrak B, Tadeusiewicz R, Karuga-Kuzniewska E, Rybak Z. Telerehabilitation approach for patients with hand impairment. Acta Bioeng Biomech. 2016;18:55–62.PubMedGoogle Scholar
  19. Levy CE, Silverman E, Jia H, Geiss M, Omura D. Effects of physical therapy delivery via home video telerehabilitation on functional and health-related quality of life outcomes. J Rehabil Res Dev. 2015;52:361–70.View ArticleGoogle Scholar
  20. Koh GC, Yen SC, Tay A, Cheong A, Ng YS, De Silva DA, Png C, Caves K, Koh K, Kumar Y, et al: Singapore Tele-technology Aided Rehabilitation in Stroke (STARS) trial: protocol of a randomized clinical trial on tele-rehabilitation for stroke patients. BMC Neurol 2015, 15:161.Google Scholar
  21. Johnson MJ, Loureiro RCV, Harwin WS. Collaborative tele-rehabilitation and robot-mediated therapy for stroke rehabilitation at home or clinic. Intell Serv Robot. 2008;1:109–21.View ArticleGoogle Scholar
  22. Lanini J, Tsuji T, Wolf P, Riener R, Novak D. Teleoperation of two six-degree-of-freedom arm rehabilitation exoskeletons. In: IEEE International Conference on Rehabilitation Robotics (ICORR). Singapore: Singapore. IEEE. p. 2015.Google Scholar
  23. Song A, Wu C, Ni D, Li H, Qin H. One-Therapist to Three-Patient Telerehabilitation Robot System for the Upper Limb after Stroke. Int J Soc Robot. 2016;8:319–29.View ArticleGoogle Scholar
  24. Andrade KO, Martins J, Caurin GA, Joaquim RC, Fernandes G. Relative performance analysis for robot rehabilitation procedure with two simultaneous users. In: IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics; Rome, Italy. IEEE; 2012.Google Scholar
  25. Caurin GA, Siqueira AA, Andrade KO, Joaquim RC, Krebs HI. Adaptive strategy for multi-user robotic rehabilitation games. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:1395–8.PubMedGoogle Scholar
  26. Gorsic M, Cikajlo I, Goljar N, Novak D. A multisession evaluation of an adaptive competitive arm rehabilitation game. J Neuroeng Rehabil. 2017;14:128.View ArticleGoogle Scholar
  27. Gorsic M, Cikajlo I, Novak D. Competitive and cooperative arm rehabilitation games played by a patient and unimpaired person: effects on motivation and exercise intensity. J Neuroeng Rehabil. 2017;14:23.View ArticleGoogle Scholar
  28. Le HH, Loureiro RCV, Dussopt F, Phillips N, Zivanovic A, Loomes MJ: Soundscape and Haptic Cues in an Interactive Painting: a Study with Autistic Children. In IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics; Rome, Italy. IEEE; 2014.Google Scholar
  29. Mace M, Kinany N, Rinne P, Rayner A, Bentley P, Burdet E. Balancing the playing field: collaborative gaming for physical training. J Neuroeng Rehabil. 2017;14:116.View ArticleGoogle Scholar
  30. Novak D, Nagle A, Keller U, Riener R. Increasing motivation in robot-aided arm rehabilitation with competitive and cooperative gameplay. J Neuroeng Rehabil. 2014;11:64.View ArticleGoogle Scholar
  31. Andrade KO, Fernandes G, Martins J, Roma V, Joaquim RC, Caurin GA. Rehabilitation robotics and serious games: An initial architecture for simultaneous players. In: Biosignals and Biorobotics Conference (BRC). Rio de Janerio, Brazil: IEEE; 2013.Google Scholar
  32. Pires FA, Santos WM, Andrade KO, Caurin GA, Siqueira AA. Robotic platform for telerehabilitation studies based on unity game engine. In: IEEE 3nd International Conference on Serious Games and Applications for Health. Rio de Janeiro, Brazil: IEEE. p. 2014.Google Scholar
  33. Tsoupikova D, Stoykov N, Kamper D, Vick R. Virtual reality environment assisting recovery from stroke. Yokohama, Japan: SIGGRAPH ASIA; 2009.View ArticleGoogle Scholar
  34. Conroy SS, Whitall J, Dipietro L, Jones-Lush LM, Zhan M, Finley MA, Wittenberg GF, Krebs HI, Bever CT. Effect of gravity on robot-assisted motor training after chronic stroke: a randomized trial. Arch Phys Med Rehabil. 2011;92:1754–61.View ArticleGoogle Scholar
  35. Ellis MD, Sukal-Moulton T, Dewald JP. Progressive shoulder abduction loading is a crucial element of arm rehabilitation in chronic stroke. Neurorehabil Neural Repair. 2009;23:862–9.View ArticleGoogle Scholar
  36. Reinkensmeyer DJ, McKenna Cole A, Kahn LE, Kamper DG. Directional control of reaching is preserved following mild/moderate stroke and stochastically constrained following severe stroke. Exp Brain Res. 2002;143:525–30.View ArticleGoogle Scholar
  37. Guadagnoli MA, Lee TD. Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning. J Mot Behav. 2004;36:212–24.View ArticleGoogle Scholar
  38. Onla-or S, Winstein CJ. Determining the optimal challenge point for motor skill learning in adults with moderately severe Parkinson's disease. Neurorehabil Neural Repair. 2008;22:385–95.View ArticleGoogle Scholar
  39. Tsoupikova D, Stoykov NS, Corrigan M, Thielbar K, Vick R, Li Y, Triandafilou K, Preuss F, Kamper D. Virtual immersion for post-stroke hand rehabilitation therapy. Ann Biomed Eng. 2015;43:467–77.View ArticleGoogle Scholar
  40. Luo X, Kenyon RV, Kamper D, DeFanti TA. On the Determinants of Size-Constancy in a Virtual Environment. The International Journal of Virtual Reality. 2009;8:43–51.Google Scholar
  41. Fischer HC, Stubblefield K, Kline T, Luo X, Kenyon RV, Kamper DG. Hand rehabilitation following stroke: a pilot study of assisted finger extension training in a virtual environment. Top Stroke Rehabil. 2007;14:1–12.View ArticleGoogle Scholar
  42. Ellis MD, Drogos J, Carmona C, Keller T, Dewald JP. Neck rotation modulates flexion synergy torques, indicating an ipsilateral reticulospinal source for impairment in stroke. J Neurophysiol. 2012;108:3096–104.View ArticleGoogle Scholar
  43. Tsoupikova D, Triandafilou K, Solanki S, Barry A, Presuss F, Kamper D. Real-time diagnostic data in multi-user virtual reality post-stroke therapy. Macau. ACM: SIGGRAPH Asia; 2016.View ArticleGoogle Scholar
  44. Cameirao MS, Badia SB, Duarte E, Frisoli A, Verschure PF. The combined impact of virtual reality neurorehabilitation and its interfaces on upper extremity functional recovery in patients with chronic stroke. Stroke. 2012;43:2720–8.View ArticleGoogle Scholar
  45. Wittmann F, Lambercy O, Gonzenbach RR, van Raai MA, Hover R, Held J, Starkey ML, Curt A, Luft A, Gassert R. Assessment-driven arm therapy at home using an IMU-based virtual reality system. In: Rehabilitation Robotics (ICORR), 2015 IEEE International Conference on. Singapore: Singapore. IEEE. p. 2015.Google Scholar
  46. Gowland C, Stratford P, Ward M, Moreland J, Torresin W, Van Hullenaar S, Sanford J, Barreca S, Vanspall B, Plews N. Measuring physical impairment and disability with the Chedoke-McMaster Stroke Assessment. Stroke. 1993;24:58–63.View ArticleGoogle Scholar
  47. Likert R. A technique for the measurement of attitudes. Archives of Psychology. 1932;22:140–55.Google Scholar
  48. Norman G. Likert scales, levels of measurement and the "laws" of statistics. Adv Health Sci Educ. 2010;15:625–32.View ArticleGoogle Scholar
  49. Barre A, Armand S. Biomechanical ToolKit: Open-source framework to visualize and process biomechanical data. Comput Methods Prog Biomed. 2014;114:80–7.View ArticleGoogle Scholar
  50. Lang CE, Macdonald JR, Reisman DS, Boyd L, Jacobson Kimberley T, Schindler-Ivens SM, Hornby TG, Ross SA, Scheets PL. Observation of amounts of movement practice provided during stroke rehabilitation. Arch Phys Med Rehabil. 2009;90:1692–8.View ArticleGoogle Scholar
  51. Iwamuro BT, Cruz EG, Connelly LL, Fischer HC, Kamper DG. Effect of a gravity-compensating orthosis on reaching after stroke: evaluation of the Therapy Assistant WREX. Arch Phys Med Rehabil. 2008;89:2121–8.View ArticleGoogle Scholar
  52. Sukal TM, Ellis MD, Dewald JP. Shoulder abduction-induced reductions in reaching work area following hemiparetic stroke: neuroscientific implications. Exp Brain Res. 2007;183:215–23.View ArticleGoogle Scholar
  53. Duncan P, Richards L, Wallace D, Stoker-Yates J, Pohl P, Luchies C, Ogle A, Studenski S. A randomized, controlled pilot study of a home-based exercise program for individuals with mild and moderate stroke. Stroke. 1998;29:2055–60.View ArticleGoogle Scholar
  54. Jurkiewicz MT, Marzolini S, Oh P. Adherence to a home-based exercise program for individuals after stroke. Top Stroke Rehabil. 2011;18:277–84.View ArticleGoogle Scholar
  55. Sluijs EM, Kok GJ, van der Zee J: Correlates of exercise compliance in physical therapy. Phys Ther 1993, 73:771–782; discussion 783-776.View ArticleGoogle Scholar
  56. Chen CY, Neufeld PS, Feely CA, Skinner CS. Factors influencing compliance with home exercise programs among patients with upper-extremity impairment. Am J Occup Ther. 1999;53:171–80.View ArticleGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement