Week 6 Discussion Response

sustainability

Review

The Intention to Use Fitness and Physical Activity Apps: A Systematic Review

Salvador Angosto 1 , Jerónimo García-Fernández 2,* , Irena Valantine 3 and Moisés Grimaldi-Puyana 2

1 Department of Physical Education and Sports, Faculty of Sports Sciences San Javier, University of Murcia, 30720 Santiago de la Ribera (Murcia), Spain; [email protected]

2 Department of Physical Education and Sports, Faculty of Educational Sciences, Universidad de Sevilla, 41013 Seville, Spain; [email protected]

3 Department of Sport and Tourism Management, Lithuanian Sports University, 44221 Kaunas, Lithuanian; [email protected]

* Correspondence: [email protected]; Tel.: +34-696-584-788

Received: 16 July 2020; Accepted: 15 August 2020; Published: 17 August 2020 !"#!$%&'(! !"#$%&'

Abstract: Recently the development of new technologies has produced an increase in the number of studies that try to evaluate consumer behavior towards the use of sports applications. The aim of this study is to perform a systematic review of the literature on the intention to use mobile applications (Apps) related to fitness and physical activity by consumers. This systematic review is a critical evaluation of the evidence from quantitative studies in the field of assessment of consumer behavior towards sport applications. A total of 13 studies are analyzed that propose models for evaluating the intentions to use fitness applications by sport consumers. The results revealed several key conclusions: (a) Technology Acceptance Model is the most widely used model; (b) the relationship between perceived utility and future intentions is the most analyzed; and (c) the most evaluated applications are diet/fitness. These findings could help technology managers to know the most important key elements to take into account in the development of future applications in sport organizations.

Keywords: physical activity; sport application; marketing consumption; technology acceptance model; smartphone app

1. Introduction

The constant technological evolution and the development of new mobile devices such as Smartphones or tablets o↵er a higher level of comfort and practical use, thus making this type of device the center of life for current consumers [1]. Globally, it is estimated that in 2019, there were 6.8 billion users worldwide and it is expected that in 2023 the number of users will increase to 7.33 billion [2]. In particular, 90% of the time dedicated to the Smartphone is for the use of mobile applications (Apps) [3].

Sustainability takes equal account of economic, environmental and social factors in any e↵ort to improve quality of life [4]. The dissemination and integration of information and communication technologies (ICT) and data management functionalities have been widely leveraged through the adoption of mobile devices, which allow people to participate in a larger way in society [5,6]. European Union (EU) policies emphasize the synergy between smart technologies and sustainable urban development because of the need for accurate, consistent and timely data for new policy formulation and the use of ICTs to facilitate service improvement [7,8].

The role of ICTs in sustainable development is clearly reflected in Goal 11 “make cities and human settlements inclusive, safe, resilient and sustainable” of the Sustainable Development Goals of the

Sustainability 2020, 12, 6641; doi:10.3390/su12166641 www.mdpi.com/journal/sustainability

Sustainability 2020, 12, 6641 2 of 24

United Nations Agenda 2030 [9], which considers ICTs as a means to advance human progress and knowledge in societies, to increase resource e�ciency, to promote economic development and protect the environment or to modernize industries on the basis of sustainable design [9,10]. Online tools and platforms contribute significantly to the repression of energy demands or pollution, promoting cities to a more environmentally sustainable economy [11]. Angleidou et al. [7] show that ICTs and the use of Apps help reduce the need for physical travel and the existence of physical workplaces.

An App is defined as “software applications usually designed to run on a Smartphone or tablet device and provide a convenient means for the user to perform certain tasks” [12] (p. 211). The increase is such that Blair [13] reported that the trade of Apps generates 189 billion dollars a year being used at least 11 times by 49% of users while 21% of the millennials open them at least 50 times a day.

Among them, health, fitness and physical activity Apps represent 5.18% of the total market [14], being used daily by 35% of people and several times a week by 40% [15]. In recent years, McKay, Wright, Shill, Stephens, and Uccellini [16] report a proliferation of Apps to improve health, including Apps to count the steps or promote physical activity in fitness centers, Apps to control diet and caloric intake or reduce poor habits such as smoking or alcohol consumption and improve mental health.

This increase in interest and number of Apps associated with physical activity could also have benefits for society. Therefore, the current situation of confinement caused by Covid-19 and consequently the reduction of physical activity, has encouraged di↵erent organizations such as the World Health Organization [17] to promote the need for physical activity at home. In fact, authors such as Banskota, Healy, and Goldberg [18] proposed di↵erent Apps as tools to maintain and improve physical and mental fitness in the Covid-19 pandemic. These Apps are linked to the fitness sector, revolutionizing the ways of doing physical activity and the relationships between fitness providers and consumers [19].

These new communication and prescription tools in sport could therefore have an impact on how organizations interact with consumers, with the appearance in recent years of studies that evaluate consumers’ motivations for using devices, the usefulness of Apps or consumers’ intentions to adopt them in di↵erent areas [20,21]. Particularly, researchers have begun to identify the factors that lead to the intention to use technologies, Smartphones and Apps in di↵erent sectors [22,23], but it is limited in the sports context.

Among the theories related to the intention to use of technologies, in the context of marketing we find the “Theory Acceptance Model” (TAM). This is the most used model by researchers to evaluate the intention to use of new technologies proposed by Davis [24]. TAM is an adaptation of the psychological theory, the “Theory of Reasoned Action” (TRA), which states that a person’s real behavior is determined by his or her intention to perform that behavior [25]. For instance, the TAM tries to explain how consumers use and accept new technologies based on two key beliefs, namely the usefulness of use and the ease of use that are predictive of consumers’ attitude towards future intention to use the new technologies [24]. Research based on TAM is one of the most widely used in professional settings because it focuses on the utilitarian aspect of the technology [26], with the intention of understanding the consumer’s intention to use it [22]. In particular, TAM has been used in di↵erent contexts such as finance, instant messaging, healthcare, gaming and tourism [27].

Although TAM has great robustness and applicability in terms of intention to use, attitude and perceived utility [27], di↵erent authors have developed new theories based on TAM such as the “Innovation and Di↵usion Theory” (IDT) [28] which considers that the user’s behavioral potential is driven by the user’s beliefs about innovation. Later, there is the “Unified Theory of Acceptance and Use of Technology Model (UTAUT)” [29] that proposes four constructs to develop TAM: performance expectation, social influence, e↵ort expectation and facilitation conditions. A second version of this model (UTAUT2) adds the constructs of hedonic motivation, price and habit, being adapted by Yuan, Ma, Kanthawala, and Peng [30] to measure the intention to use of health and fitness Apps. In addition, in sport, the “Sport Website Acceptance Model” (SWAM) is proposed by Hur, Ko, and Claussen [31] and is based on a framework of understanding how sport fans perceive and accept the websites of

Sustainability 2020, 12, 6641 3 of 24

their sport teams, how their level of participation and commitment to the sport team influences the intention to use the website and the actual consumption behavior they ultimately perform. Based on these models and theories, in recent years researchers have paid attention to the intention to use new technologies in di↵erent contexts such as e-payment, e-government, e-banking, retail or education [32–36]. Similarly, in academic sports literature there is also an increasing attention to the behavior of fans and consumers, with studies with di↵erent approaches such as motivation on sports websites [31], loyalty [37], participation, commitment and attributes [38,39], marketing opportunities [40], intention to use sports wearable [41], consumption of Smartphones and sports Apps [42], sports team Apps [1], fitness Apps [43,44] and sports products [45,46].

However, existing research does not provide clear results on what factors drive sports fans or consumers to use Smartphones or Apps and to benefit from new forms of experiences in sport [42]. In fact, the factors influencing the intention to use Smartphones and Apps di↵er depending on the types of products consumed and the marketing implications [47]. Therefore, while studies have been conducted on the intentions of use of technology, Smartphones and fitness Apps, there is not a review that captures the main findings of these studies. For this reason, the aim of this study is to conduct a systematic review of the literature on consumers’ intention to use Apps related to fitness and physical activity by consumers.

2. Materials and Methods

2.1. Search Strategy

The search terms for Smartphone use, Fitness and Sport Apps represented the concepts of App, Physical Activity and Use, with the search strategy for the di↵erent databases presented in Table 1. Di↵erent databases were selected to include a wide range of areas related to this interdisciplinary study, including sports science, marketing, health and psychology. The databases used were Web of Science, Scopus, SPORTDiscus (EBSCO), PsycINFO (Ovid), ABI/Inform (Ovied) and MEDLINE (Pubmed). The search was conducted between 18 March 2019 and 4 August 2020. The search covered all years and no limitations were placed on document type and language.

Table 1. Database search strategy.

Category Search Terms

App (“Smart phone *” or Smartphone * or smart-phone * or “cell * phone” or “cell-phone *” or “mobile phone *” or “mobile-phone” or “mobile device” or “mobile telephone” or * phone or Android * or iOS or app or apps or “mobile application *” or application)

Physical Activity

(“physical activit *” or exercise * or “active living” or walk * or “active transport” or “leisure activit *” or fitness or sport or “sport *” or “weight maintenance” or “maintaining weight” or “weight management”)

Use (“intention to use” or “app * usage” or “intent * to use” or usage or “behavioral intention *” or “behavior * change” or usability or “attitude toward” or consumption or Technology Acceptance Model)

Combination 1 and 2 and 3

* Truncation operator: word-based search.

2.2. Inclusion and Exclusion Criteria

For the purposes of this review, we included empirical papers in peer-reviewed journals, excluding dissertations and abstracts. Grey literature was not included, ruling out evaluation reports, annual reports, articles in nonpeer reviewed journals and other means of publication. The inclusion criteria for the articles in the search were: (i) journal articles; (ii) publications in English; (iii) use of any type of mobile application in the sports and fitness context; and (iv) measurement of the intention to use the App through a questionnaire. As exclusion criteria have been used: (i) Congress proceedings, book chapters, books or other types of publications; (ii) no mobile Apps were used in the sports context,

Sustainability 2020, 12, 6641 4 of 24

(iii) theoretical studies, qualitative approach or reviews; (iv) articles in a language other than English; and (v) duplicate articles.

2.3. Assessment of Methodological Quality

The risk of bias was assessed using a 20-item tool adapted by the authors to the context of sports marketing study typology in which there are no intervention processes on the subjects of the Consolidated Standards of Reporting Trials (CONSORT) checklist [48]. Each study was independently scored by two reviewers evaluating the di↵erent sections that make up the studies and scoring each item with 1 if the study satisfactorily met the criterion, and with 0 if the study did not satisfactorily meet the criterion or if the item was not applicable to the study. Disagreements between the reviewers were resolved by checking and discussing the original study until consensus was reached. Reviewer A is a researcher with extensive experience specializing in the field of sports management, fitness centers and development of new technologies. Reviewer B is a predoctoral fellow in sports management with focus research on methodological and statistical aspects. The results of assessment of methodological quality were shown in Appendix A.

2.4. Data Extraction and Synthesis

Figure 1 shows the Flow Diagram proposed by Moher, Liberati, Tetzla↵, and Altman [49] following the PRISMA methodology in all points that could be common to a systematic review of these characteristics. The initial database search returned 113,537 results, reduced to 36,105 once duplicates were eliminated. One reviewer conducted a full scan of the title, then an abstract review and finally a full text review using the inclusion and exclusion criteria. Among the articles that remained at the abstract level (n = 4), a second reviewer also examined the abstracts of the articles to confirm their eligibility, and there were no discrepancies with the first reviewer.

Sustainability 2020, 12, x FOR PEER REVIEW 4 of 25

any type of mobile application in the sports and fitness context; and (iv) measurement of the intention to use the App through a questionnaire. As exclusion criteria have been used: (i) Congress proceedings, book chapters, books or other types of publications; (ii) no mobile Apps were used in the sports context, (iii) theoretical studies, qualitative approach or reviews; (iv) articles in a language other than English; and (v) duplicate articles.

2.3. Assessment of Methodological Quality

The risk of bias was assessed using a 20-item tool adapted by the authors to the context of sports marketing study typology in which there are no intervention processes on the subjects of the Consolidated Standards of Reporting Trials (CONSORT) checklist [48]. Each study was independently scored by two reviewers evaluating the different sections that make up the studies and scoring each item with 1 if the study satisfactorily met the criterion, and with 0 if the study did not satisfactorily meet the criterion or if the item was not applicable to the study. Disagreements between the reviewers were resolved by checking and discussing the original study until consensus was reached. Reviewer A is a researcher with extensive experience specializing in the field of sports management, fitness centers and development of new technologies. Reviewer B is a predoctoral fellow in sports management with focus research on methodological and statistical aspects. The results of assessment of methodological quality were shown in Appendix A.

2.4. Data Extraction and Synthesis

Figure 1 shows the Flow Diagram proposed by Moher, Liberati, Tetzlaff, and Altman [49] following the PRISMA methodology in all points that could be common to a systematic review of these characteristics. The initial database search returned 113,537 results, reduced to 36,105 once duplicates were eliminated. One reviewer conducted a full scan of the title, then an abstract review and finally a full text review using the inclusion and exclusion criteria. Among the articles that remained at the abstract level (n = 4), a second reviewer also examined the abstracts of the articles to confirm their eligibility, and there were no discrepancies with the first reviewer.

Figure 1. PRISMA flow diagram. Source: Moyer et al. [49].

Sustainability 2020, 12, 6641 5 of 24

A form was developed for data extraction that included the following aspects: (a) year of publication; (b) country of study; (c) number of participants; (d) gender; (e) age of participants; (f) type of application evaluated; (g) theory used; (h) analyses performed; (i) variables included; and (j) main results. In order to homogenize the results of the di↵erent studies and to make the data more homogeneous, the confidence intervals of each correlation (CI 95%) and the e↵ect size with its confidence intervals (CI 95%) of each relationship were calculated through the Fisher’s Z statistics [50].

3. Results

3.1. Analysis of the Risk of Bias in Studies

To test quality, risk of bias analysis of the 19 studies evaluated in the research showed that only three studies had a high score of 15 points or more out of 20 total [1,45,51], most studies (n = 14) had a mean score between 10 and 15 points and only two studies had a score below 10 points [52,53]. It should be noted that none of the studies analyzed carried out a calculation of the sampling required for the generalization of the results, which could be due to the fact that all the studies carried out a selection of the sample for convenience within a certain population. There are also few studies that established criteria for inclusion in the sample to be selected (n = 5) and no study indicates the author who carried out each part of the research.

3.2. Summary of Reported Intervention Outcomes

Results of the descriptive data from the analysis of the articles can be seen in Table 2. The analysis shows that this topic is very recent within the context of sports marketing, with only 13 quantitative studies addressing the intention to use of sports applications by the sports consumer through the use of self-administered questionnaires and online. Of the articles analyzed, the majority were published in 2018 (n = 5) and 2020 (n = 5), followed by those published in 2017 (n = 4), three articles were published in 2015, while only one article was found in 2016 and 2019. Korea has been the country with the highest production with six articles, followed by the United States and Hong Kong with three publications, China had two studies and other countries such as Germany, India, Iran, South Africa and Taiwan each had one publication.

Analyzing the sample used in the di↵erent studies, there is a total of 16,025 subjects with an average sample of 843.42 subjects per study, with the Ndayizigamiye; Kante, and Shingwenyana study [54] having the smallest sample (n = 139) and Wei, Vinnikova, Lu and Xu study [55] having the largest sample with a total of 8840 subjects. Approximately a half of the studies (n = 8) used university students as a sample, followed by studies that considered users of sports applications (n = 4) and other studies took as their general population [54–56], a population of sports consumers [45], employees of a sports organization [57] and members of a fitness community [44,58]. Most studies had a higher proportion of females than males (n = 9), followed by studies that had parity in the sample (n = 5), four studies had a higher proportion of males while one study did not indicate the gender distribution of the sample [34]. Finally, all studies except Ha et al. [42] and Yoo et al. [53] reported some data on the age of the subjects. About half of the studies (n = 10) expressed age using a range, five studies showed age using mean (M = 24.58 years) and two studies did not specify the age [55,59]. The analysis indicated that mainly the study population are young subjects between 20 and 29 years old and all are over 18 years old except Lee, Kim and Wang [45] which also included 17-year-old subjects. Li, Liu, Ma and Zhang [46] sampled subjects over 25 years of age, while Huang and Ren [60] and Mohammadi and Isanejad [57] were at least 30 years old.

Sustainability 2020,12,6641

6 of24

Table 2.D

escriptive data

ofthe analysis

ofthe selected

studies.

A uthors

C ountry

Sam ple

A pp

Type T

heory D

ata A

nalysis M

ethods M

easure O

utcom es

Beldad &

H egner

[43] G

erm any

G erm

an’s app

user (n =

476) M

ale:50.0% Fem

ale:50.0% A

ge: 26.7±

5.0

Sport inform

ation TA

M C

ontentA nalysis

Trustin the

Fitness A

pp D

eveloper;D escriptive

SocialN orm

;Injunctive SocialN

orm ;Perceived

Ease ofU

se;Perceived U

sefulness;Intention to

C ontinue

U sing

a Fitness

A pp

Byun,C hiu,&

Bae [45]

K orea

K orean

consum ers

(n =

261) M

ale:4 9.1%

;Fem ale:50.9%

A ge:20–29

(29.9% );30–39

(34.9% );40+

(5.2% )

SportBrand TA

M C

ontentA nalysis

Perceived Enjoym

ent;Perceived Ease

ofU se;

Perceived U

sefulness;Intention to

use;A ctual

usage

C hen

& Lin

[44] Taiw

an

Fitness C

om m

unity (n =

994) A

ge:20� (10.06%

);20–29 (56.14%

);30–39 (1.83%

);40–49 (8.65%

);50–59 (3.32%

)

D iet/Fitness

TR A

M C

ontentA nalysis

H ealth

C onsciousness;O

ptim ism

;Innovativeness; D

iscom fort;Insecurity;Perceived

Ease ofU

se; Perceived

U sefulness;A

ttitude tow

ard U

sing A

pp;Intention to

dow nload

app

C hiu

& C

ho [61]

H ong

K ong

K orean

university students

(n =

204) M

ale:51.9% ;Fem

ale:48.1% A

ge:19–25 (71.8%

);26–30 (10.7%

);30+ (17.5%

)

H ealth

/Fitness TR

A M

D escriptive

C ontentA

nalysis

O ptim

ism ;Innovativeness;Insecurity;

D iscom

fort;Perceived U

sefulness;Perceived Ease

ofU se;Perceived

Enjoym ent;Intention

to use

C hiu,C

ho,& C

hi[56] H

ong K

ong

C hinese

population (n =

342) M

ale:45.6% ;Fem

ale:54.4% A

ge:20� (1.2%

);21–25 (14.9%

); 26–30

(35.4% );31–35

(29.8% );

36–40 (11.1%

);40+ (7.6%

)

H ealth

/Fitness EC

M D

escriptive C

orrelational C

ontentA nalysis

Investm entsize;Q

uality ofalternative;

C om

m itm

ent;C onfirm

ation ofexpectations;

Satisfaction;Perceived U

sefulness;C ontinuance

Intention

C ho,Lee,K

im ,&

Park [59]

K orea

U niversity

students (n =

294) M

ale:33.0% Fem

ale:67.0% A

ge:23.2 D

iet/Fitness TA

M C

orrelational C

ontentA nalysis

A ppearance

Evaluation;Fitness Evaluation;

A ppearance

O rientation;Fitness

O rientation;

Perceived U

sefulness;Intention to

U se

A pp

C ho,Lee,&

Q uinlan

[51] K

orea U

niversity students

(n =

508) M

ale:34.6% ;Fem

ale:65.4% A

ge:21.5 D

iet/Fitness TA

M D

escriptive C

ontentA nalysis

Subjective N

orm s;Entertainm

ent;R ecordability;

N etw

orkability;Perceived Ease

ofU se;Perceived

U sefulness;BehavioralIntention

to U

se

C ho

& K

im [52]

K orea

U niversity

students (n =

277) M

ale:34.3% ;Fem

ale:65.7% A

ge:22.5 D

iet/Fitness TA

M C

ontentA nalysis

Sm artphone

U se

E �

cacy;InternetInform ation

U se

E �

cacy;InternetInform ation

C redibility;

Perceived Ease

ofU se;Perceived

U sefulness;

BehavioralIntention

D him

an,A rora,

D ogra,&

G upta

[58] India

Indian fitness

lefts users

(n =

324) M

ale:54.0% ;Fem

ale:46.0% A

ge:20� (16.0%

);20–40 (80.0%

); 40+

(4.0% )

Fitness U

TA U

T2 D

escriptive C

orrelational C

ontentA nalysis

Perform ance

Expectancy;E ↵ortExpectancy;Self

E �

cacy;SocialInfluence;Facilitating C

onditions; H

edonic M

otivation;Price Value;Personal

Innovativeness;H abit;BehavioralIntention

Sustainability 2020,12,6641

7 of24

Table 2.C

ont.

A uthors

C ountry

Sam ple

A pp

Type T

heory D

ata A

nalysis M

ethods M

easure O

utcom es

H a,K

ang,& K

im [42]

K orea

U niversity

students (n =

226) M

ale:50.8% ;Fem

ale:49.2% A

ge:25.3

Sport Inform

ation TA

M D

escriptive C

ontentA nalysis

SportInvolvem ent;SportC

om m

itm ent;Social

Influence;PersonalA ttachm

ent;M edia

M ultitasking;Perceived

Enjoym ent;Perceived

Ease ofU

se;Perceived U

sefulness;U sage

Intention

H uang

& R

en [60]

H ong

K ong

C hinese

app users

(n =

449) M

ale:43.0% ;Fem

ale:57.0% A

ge:31.85± 6.9

Fitness TA

M R

egression

Instruction Provision;Self-M

onitoring; Self-R

egulation;G oalA

ttainm ent;Exercise

Self E �

cacy;Perceived U

sefulness;Perceived Ease

of U

se;Perceived Enjoym

ent;C ontinuance

Intention

K im

,K im

,& R

ogol[1] U

nited States

A pp

users (n =

233) M

ale:68.2% Fem

ale:31.8% A

ge:18–24 (46.8%

);25–34 (31.8%

);35–44 (13.7%

);45–54 (7.3%

);55+ (0.4%

)

SportTeam TA

M D

escriptive C

ontentA nalysis

Innovativeness;Perceived Ease

ofU se;Perceived

Enjoym ent;Perceived

Trust;Perceived U

sefulness; Intention;SportA

pps U

se

Lee,K im

,& W

ang [62]

U nited

States

C ollege

students (n =

267) M

ale:32.2% Fem

ale:67.8% A

ge:17–20 (48.3%

);21–25 (40.8%

);26–29 (8.2%

);29+ (2.6%

)

SportA pp

U TA

U T

C orrelational

C ontentA

nalysis

Entertainm entM

otivation;SocialU tility

M otivation;Perform

ance Expectancy;E

↵ort Expectancy;SocialInfluence;Intention

to M

obile Sports

A pps

U se

Li,Liu,M a,&

Z hang

[63] C

hina

SportA pp

users (n =

211) M

ale:45.02% Fem

ale:54.98% A

ge:25–30 (41.71%

);30–35 (47.87%

)35+ (10.43%

)

Social Fitness-tracking

U TA

U T2

C ontentA

nalysis A

ctivity A

m ountR

anking;A ctivity

Frequency R

anking;C onfirm

ation;U pw

ard C

om parison

Tendency;C ontinuous

Intention

M oham

m adi&

Isanejad [57]

Irán

Em ployers

SportO rganization

(n =

332) M

ale:37.3% Fem

ale:62.7% A

ge:30� (10.0%

);31–40 (44%

); 41–50

(38% );50+

(8% )

IT inform

ation TA

M D

escriptive C

orrelational C

ontentA nalysis

Technology A

nxiety;Technology Self-e�

cacy; Perceived

Enjoym ent;Perceived

Ease ofU

se; Perceived

U sefulness;U

ser Satisfaction;A

ttitude; Intention

to use

N dayizigam

iye; K

ante,& Shingw

enyana [54]

South A

frica

South A

frican population

(n =

139) M

ale:41.5% ;Fem

ale:58.5% A

ge:18–23 (57.15%

);24–29 (29.9%

);30–35 (7.5%

)

m H

ealth U

TA U

T C

orrelational C

ontentA nalysis

A w

areness,E ↵ortExpectancy;Facilitating

C onditions;Perform

ance Expectancy;Social

Influence;BehavioralIntention

Sustainability 2020,12,6641

8 of24

Table 2.C

ont.

A uthors

C ountry

Sam ple

A pp

Type T

heory D

ata A

nalysis M

ethods M

easure O

utcom es

W ei,V

innikova,Lu,& X

u [55]

C hina

C hinese

population (n =

8840) M

ale:4.78% ;Fem

ale:74.55% D

iet/Fitness U

TA U

T D

escriptive C

orrelational C

ontentA nalysis

Perceived Benefits;Perceived

Barriers;Perceived Threats;Self-E

� cacy;R

isk Perception;

Perform ance

Expectancy;W eightLoss

Intention; BehavioralIntention;U

se Behavior

Yoo,K o,&

Yeo [53]

K orea

U niversity

students (n =

1331) M

ale:65.9% ;Fem

ale:34.1% SportC

ontent TA

M C

ontentA nalysis

Perceived Trust;Perceived

U sefulness;A

ttitude; U

sing intention

Yuan,M a,

K hantaw

ala &

Peng [30]

U nited

States

U niversity

students (n =

317) M

ale:21.1% Fem

ale:78.9% A

ge:21 D

iet/Fitness U

TA U

T2 C

ontentA nalysis

Perform ance

expectancy;E ↵ortexpectancy;Social

influence;Facilitating conditions;Price

value; H

edonic m

otivation;H abit;Intention

to use

EC M

:Expectation-C onfirm

ation M

odel.

Sustainability 2020, 12, 6641 9 of 24

Regarding the type of App evaluated, six studies evaluated the intention to use diet and fitness applications [30,44,51,52,55,59], another five studies evaluated sports information Apps [42,43,53,57,62], two studies measured the intention of fans to use the sports team app [1,45], health and fitness app [56,61], or fitness [58,60] and one study evaluated a social fitness-tracking app [63] and mHealth related to promote physical activity [54]. The most widely used theory for the design and use of the mobile sports app intent of use assessment instrument was TAM (n = 10). Chen and Lin [44] and Chiu and Cho [61] used a variant of the TAM, Theory of Readiness and Acceptance Model (TRAM), three studies used UTAUT [54,55,62] and UTAUT2 [30,58,63], and an article with the Expectation-Confirmation Model (ECM) [56]. The most common method of analysis used was a content analysis by structural equations using the AMOS statistical package (n = 9) and the rest of the Partial Least Square studies. Seven studies also performed a correlation analysis of the data [54–59,62], and eight studies performed descriptive analysis in addition to content analysis [1,42,51,55–58,61].

The variables used by the di↵erent studies have been very varied, where the intention to use App has been found in all studies as a common factor. Considering that this systematic review study focused on studies based on TAM as the most commonly used theory in sports marketing studies, it implies that there are other common variables among most studies such as perception of usefulness (n = 13) and perception of ease of use (n = 10). Some studies have included other di↵erent perceptions by relating them to the previous ones and the intention to use, such as the perception of enjoyment [1,42,45,60,61] or the perception of trust [1,53]. The remaining variables used have been very diverse, with each study using di↵erent variables that can be seen in Table 2. However, the variable of social influence has received greater interest from researchers and has been considered in five studies [30,42,54,58,62].

Analyzing the quantitative data on the relationships between the most common variables associated with TAM (Table 3), the six studies that had a di↵erent theory such as UTAUT [54,55,62] or UTATUT2 [30,58,63] were excluded; however, the study that used ECM was included because there was a relationship between variables “perception usefulness” and “intention to use” [56]. The sample was very heterogeneous in terms of the results of the existing relationships and sample size in each study. In order to homogenize these results, the e↵ect size of each correlation was calculated. A total of seven relationships were identified between the di↵erent variables associated with TAM such as perception of ease of use (PEOU), perception of utility (PU), perception of enjoyment (PE), perception of trust (PT), intention to use (ITU) and actual usage (AU).

Sustainability 2020,12,6641

10 of24

Table 3.Q

uantitative data

on the

relationships betw

een “Theory

A cceptance

M odel”

(TA M

)variables in

the selected

studies.

A uthors

PE-PEO U

PEO U

-PU PE-IT

U PEO

U -IT

U PU

-IT U

PT-IT U

IT U

-A U

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

R (C

I95% )

Beldad &

H egner

[43] –

0.180 ***

(0.092;0.266) –

0.440 ***

(0.365;0.510) 0.330

*** (0.246;0.408)

– –

Byun,C hiu,&

Bae [45]

0.584 ***

(0.498–0.659) 0.583

*** (0.497;0.658)

0.454 ***

(0.352;0.545) 0.157

* (0.036;0.273)

0.318 ***

(0.205;0.423) –

0.306 ***

(0.19;0.41)

C hen

& Lin

[44] –

0.650 ***

(0.613;0.685) –

0.310 ***

(0.253;0.365) 0.510

*** (0.463;0.555)

– 0.530

*** (0.484;0.573)

– 0.660

*** (0.623;0.694)

– 0.300

*** (0.24;0.36)

0.500 ***

(0.452;0.545) –

0.520 ***

(0.473;0.546)

C hiu

& C

ho [61]

0.453 ***

(0.337;0.556) 0.222

*** (0.087;0.349)

0.293 ***

(0.162;0.414) 0.213

** (0.078;0.340)

0.373 ***

(0.248;0.486) –

–

C hiu,C

ho,& C

hi[56] –

– –

– 0.299

** (0.199;0.392)

– –

C ho,Lee,K

im ,&

Park [59]

– –

– –

0.750 ***

(0.70;0.80) –

–

C ho,Lee,&

Q uinlan

[51] –

0.229 ***

(0.145;0.310) –

0.001 (�

0.086;0.088) 0.431

*** (0.357;0.500)

– –

C ho

& K

im [52]

– 0.580

*** (0.496;0.653)

– �

0.130 (�

0.244;� 0.012)

0.800 ***

(0.753;0.839) –

–

H a,K

ang,& K

im [42]

0.770 *

(0.711–0.818) 0.870

* (0.834;0.899)

� 0.08

(� 0.51;0.208)

0.350 *

(0.230;0.460) 0.090

(� 0.041;0.218)

– –

H uan

& R

en [60]

– –

0.233 ***

(0.144;0.319) 0.270

*** (0.182;0.354)

0.404 ***

(0.324;0.479) –

–

K im

,K im

,& R

ogol[1] –

–

0.313 **

(0.192;0.424) 0.163

(0.035;0.286) 0.628

*** (0.543;0.700)

0.387 *

(0.272;0.491) 0.787

*** (0.733;0.831)

0.331 ***

(0.212;0.440) 0.204

(0.078;0.324) 0.728

*** (0.661;0.783)

0.475 *

(0.369;0.569) 0.806

*** (0.756;0.847)

0.287 **

(0.165;0.400) 0.149

(0.021;0.272) 0.580

*** (0.489;0.660)

0.359 *

(0.241;0.466) 0.808

*** (0.758;0.848)

M oham

m adi&

Isanejad [57]

0.370 **

(0.273–0.459) 0.386

** (0.290;0.473)

– 0.173

** (0.067;0.283)

0.192 **

(0.086;0.293) –

–

Yoo,K o,&

Yeo [53]

– –

– –

0.454 ***

(0.410;0.496) 0.403

*** (0.357;0.447)

0.963 ***

(0.959;0.967)

Sustainability 2020,12,6641

11 of24

Table 3.C

ont.

A uthors

Fisher’s Z

(C I95%

) Fisher’s

Z (C

I95% )

Fisher’s Z

(C I95%

) Fisher’s

Z (C

I95% )

Fisher’s Z

(C I95%

) Fisher’s

Z (C

I95% )

Fisher’s Z

(C I95%

)

Beldad &

H egner

[43] –

0.18 (0.09;0.27)

– 0.47

(0.38;0.56) 0.34

(0.25;0.43) –

– Byun,C

hiu,& Bae

[45] 0.67

(0.55;0.79) 0.67

(0.55;0.79) 0.49

(0.37;0.61) 0.16

(0.04;0.28) 0.33

(0.21;0.45) –

0.32 (0.19;0.44)

C hen

& Lin

[44] –

0.78 (0.71;0.84)

– 0.32

(0.26;0.38) 0.56

(0.50;0.63) –

0.59 (0.53;0.65)

– 0.79

(0.73;0.86) –

0.31 (0.25;0.37)

0.55 (0.49;0.61)

– 0.58

(0.51;0.64) C

hiu &

C ho

[61] 0.49

(0.35;0.63) 0.23

(0.09;0.36) 0.30

(0.16;0.44) 0.22

(0.08;0.35) 0.39

(0.25;0.53) –

– C

hiu,C ho,&

C hi[56]

– –

– –

0.31 (0.20;0.41)

– –

C ho,Lee,K

im ,&

Park [59]

– –

– –

0.97 (0.86;1.09)

– –

C ho,Lee,&

Q uinlan

[51] –

0.23 (0.15;0.32)

– 0.001

(� 0.09;0.09)

0.46 (0.37;0.55)

– –

C ho

& K

im [52]

– 0.66

(0.54;0.78) –

� 0.13

(� 0.25;�

0.01) 1.10

(0.98;1.21) –

– H

a,K ang,&

K im

[42] 1.02

(0.89;1.15) 1.33

(1.20;1.46) 0.08

(� 0.05;0.21)

0.37 (0.23;0.50)

0.09 (�

0.04;0.22) –

– H

uan &

R en

[60] –

– 0.24

(0.14;0.33) 0.28

(0.18;0.37) 0.43

(0.34;0.52) –

–

K im

,K im

,& R

ogol[1] –

– 0.32

(0.19;0.45) 0.16

(0.04;0.30) 0.74

(0.61;0.87) 0.41

(0.28;0.54) 1.06

(0.93;1.19) 0.34

(0.214;0.47) 0.21

(0.08;0.34) 0.92

(0.80;1.05) 0.52

(0.39;0.65) 1.12

(0.99;1.24) 0.30

(0.16;0.42) 0.15

(0.02;0.28) 0.66

(0.53;0.79) 0.38

(0.25;0.51) 1.12

(0.99;1.25) M

oham m

adi& Isanejad

[57] 0.39

(0.28;0.50) 0.41

(0.30;0.51) –

0.17 (0.07;0.28)

0.19 (0.09;0.30)

– –

Yoo,K o,&

Yeo [53]

– –

– –

0.49 (0.44;0.54)

0.43 (0.37;0.48)

1.99 (1.93;2.04)

N ote:*p

0.05;**p 0.01;***p

0.001;PE:Perceived Enjoym

ent;PEO U

:Perceived Ease

ofU se;PT:Perceived

Trust;PU :Perceived

U sefulness;ITU

:Intention to

U se;A

U :A

ctualU sage.

Sustainability 2020, 12, 6641 12 of 24

The first relationship between the variables PE and PEOU was found in four di↵erent studies showing results of di↵erent degree of relationship, although all significant. In the study by Mohammadi and Isanejad [57], the e↵ect size was found to indicate a low-medium influence of PE on PEOU [Fisher’s Z = 0.39 (0.2; 0.50)], Chiu and Cho [61] found a medium influence [Fisher’s Z = 0.49 (0.35; 0.63)], while Byun et al. [45] found a medium-high influence of PE on PEOU [Fisher’s Z = 0.67 (0.55; 0.79)] and a very high influence of enjoyment in students on ease of use [Fisher’s Z = 1.02 (0.89; 1.15)]. Another relationship of perceptions and the most studied is between PEOU and PU. Although all the existing relationships were significant, they had di↵erent levels of relationship: for example, three studies had a low influence of PEOU on PU with values lower than 0.30 [43,51,61], while Mohammadi and Isanejad [57] found that PEOU had an influence on lower-middle on PU [Fisher’s Z = 0.41 (0.30; 0.54)]. Other studies found a high or very high influence between these two variables [Fisher’s Z� 0.66]. In the selected studies, no other studies were found where the di↵erent dimensions of TAM were interrelated.

The relationship of the di↵erent variables on the ITU is explained below. Firstly, four studies found an influence of PE on ITU where PE had a low influence [60,61], and moderate and significant influence on ITU in the di↵erent models evaluated with an e↵ect size range between 0.30 and 0.49 [1,45]. In contrast, Ha et al. [42] did not find that PE had an influence on ITU [Fisher’s Z = 0.08 (�0.05; 0.21)]. One of the most studied relationships is the PEOU with ITU, being tested in ten studies with contradictory results. Seven studies found a significant relationship, however, except Beldad and Hegner [43] which had a moderate influence [Fisher’s Z = 0.47 (0.38, 0.56)] and Chen and Lin [44] in their study evaluated two models with moderate-low influences [Fisher’s Z1 = 0.31 (0.25, 0.37); Fisher’s Z2 = 0.32 (0.26, 0.38)]. The rest had a low influence [45,57,60,61] or nonsignificant influence [1,51,52].

The relationship between PU and ITU was evaluated in all of the studies analyzed finding di↵erent levels of relationship in the studies. Most studies (n = 9) found a moderate and significant influence between both variables with a range of Fisher’s Z = 0.31–0.66 [1,43–45,51,53,56,60,61]. Some studies found a significant high to very high ratio with a large influence of PU on ITU based on e↵ect size values above 0.70 [52,59] and in two of the three models proposed by Kim et al. [1]. Mohammadi and Isanejad [57] found a low influence on sports organization employees [Fisher’s Z = 0.19 (0.09, 0.30) and Ha et al. [42] found no PU influence on ITU in university students on sports information Apps.

In turn, two studies evaluated the relationship between PU and ITU observing the existence of a moderate and significant influence between both variables in the three models proposed by Kim et al. [1] [Fisher’s Z = 0.38–0.52] and Yoo et al. [53] [Fisher’s Z = 0.43 (0.37; 0.48)]. Finally, some studies evaluated the relationship between ITU and AU found relationships where ITU had a moderate-low influence [45] [Fisher’s Z = 0.32 (0.19, 0.44)], while other studies found a moderate influence [44] and a very high influence [1,53].

3.3. Summary of Factors Relationship by Age

In order to carry out the analysis of the di↵erent relationships between the factors analyzed (Table 3) based on age, an analysis was made of the data shown by the di↵erent studies. The studies have been classified into three groups considering the mean age or the age range with the highest percentage. One group was established with six studies that had a higher proportion of subjects under 25 years [1,51–53,59,61], a second group formed by three studies with the age of the subjects between 25 and 30 years [42,43,56] and finally, a third group had a higher proportion of subjects over 30 years [45,57,60]. Chen and Lin’s study [44] was not classified as it indicated that their sample was mostly composed of subjects between 20 and 30 years of age.

The relationship between PE and PEOU showed that the study of group 25–30 years was very high and significant [42], compared to group under 25 and group over 30, which had very similar moderate and significant relationships with an e↵ect size range between 0.39 and 0.67 [45,57,61]. The relationship between PEOU and PU found that the studies with the lowest influence [43] and the highest influence [42] belonged to the group 25–30. The studies of the group over 30 had generally more influencing than the group under 25 who had the most studies with a low influence. However,

Sustainability 2020, 12, 6641 13 of 24

the study by Chen and Lin [44] reported that in their two models the relationship between PEOU and PU was high above 0.70, and significant. Concerning the PE relationship on ITU, the group over 30 had a moderate and significant overall influence [45,60], being higher than the group under 25 [1,61], while the study of the group 25–30 found no relationship between these two factors [42].

The group 25–30 revealed a significant influence of moderate level on the relationship between PEOU and ITU with e↵ect sizes higher than 0.37 [42,43], and the studies of the group over 30 found the existence of significant relationships with a low influence [45,57,60]. On the other hand, most studies of the group under 25 did not show the existence of significant di↵erences between these factors [1,51,52], except the study of Chiu and Cho [62] that had a low and significant influence. Chen and Lin [44] obtained moderate-low relations in their two models in the population between 20 and 30 years old. The most studied relationship in all studies was between PU and ITU, with a great variability in the results according to the di↵erent studies in each age group. Therefore, the larger influence has been found in the studies of the group under 25, with studies that had moderate [51,53,60], high [1,59] and very high [52] relationships, while one study of the group 25–30 observed a high influence [59] and two studies showed moderate influences [43,56]. However, the group 25–30 had one study in which no relationship was found between PU and ITU [42]. The group over 30 had all significant relationships with a variable low level [57] to moderate level [45,60]. In addition, Chen and Lin [44] reported a moderate influence of PU on UTI in their two proposed models. Finally, the relationship between ITU and AU has not been studied much, and no studies were found that took it into account in the group 25–30. Regarding the other two groups, the group under 25 obtained that the youngest ones presented a great influence between ITU and AU giving them significant and high relationships with an e↵ect size above one point [1,53]. In the study of group over 30, a moderate-low relationship was found [45], and Chen and Lin had moderate-significant relationships [44].

4. Discussion

The continuous technological advances have awakened the interest of marketing researchers in the intention to use Apps, especially in the field of sports. Walter [64] explained the existence of a trend towards increased interest by fitness consumers in using Apps for exercise control. Therefore, the aim of this study was to conduct a systematic review of the literature on consumers’ intention to use Apps related to fitness and physical activity. The result of the systematic search has been the existence of a remarkable interest in the subject, as the studies were found in the last four years; however, studies focused on sport Apps are still limited (n = 19) with very heterogeneous methodologies.

In the context of marketing, the existence of di↵erent theories to explain sports consumer behavior has been observed, with TAM being the most applied in the di↵erent studies found (n = 10) or some version derived from it such as TRAM [44,61], UTAUT in the study carried out by Lee et al. [62], Ndayizigamiye et al. [54] and Wei et al. [55], or UTAUT2 used by Dhiman et al. [58], Li et al. [46] and Yuan et al. [20]. One study used a di↵erent model, ECM, which was not derived from TAM [56]. Although most publications use the same base, no similar studies have been found that can be compared since the di↵erent authors have used the two basic constructs (PEOU and PU) of the theory proposed by Davis [24] and have incorporated other variables such as PE [1,42,45,57,60,61] or PT [1,53]. In addition, they have been relating these variables to the Apps ITU and AU [1,44,45,53] or to various other variables outside of TAM such as social influence [30,42,54,58,62], Health Consciousness [44], social norms [43] or sports context as sport involvement or sport commitment [42].

Most of the studies analyzed have tried to predict the influence of PEOU and PU on ITU, finding significant relationships between both constructs [42–45,51,52,57,61]. These relationships have been evaluated previously in the context of sport websites as a technological tool prior to the appearance of the Apps [31,38,39,65]. However, the findings found are di↵erent from those proposed by di↵erent studies that concluded with a greater influence of PU on PEOU in ITU [24,38,49,65–70] or where PEOU had no significant influence on ITU [1,51,52] or in the case of Ha et al. [42] where they found that PU

Sustainability 2020, 12, 6641 14 of 24

had no influence on ITU as opposed to PEOU. In particular, it should be noted that Davis [24] argued that PEOU is a secondary determinant construct of ITU in the perception of technology.

Furthermore, the use of other added beliefs such as PE or PT can also play a key role within the TAM, as PE not only has influence on PEOU [42,45,57,61] but also on ITU [1,45,60,61]. Ha et al. [42] found no influence of PE on ITU, but the authors felt that while it was not related to ITU, it was related to PEOU, allowing sports consumers to undermine the di�culty of using the technology [71]. These results highlight the hedonic motives of sports consumers who often overlook entertainment [26,65,71] and may make the App more interesting to sports fans if they find it fun and easy to use. On the other hand, PT has been found to be an important and influential factor on ITU [1,38,39,53,69]; this importance may be motivated by the fact that if a sports fan finds the information they receive useful and reliable they will show a greater interest in continuing to use the technology.

For example, in a study in Germany on the use of fitness Apps, it showed that the reasons for using them were to have the achievement of fitness goals and the enjoyment of being able to share the results obtained with their contacts [72], so gamification could have a direct relationship with their use [73]. Another aspect to take into account for consumers is that if they find the App tedious and requiring tedious procedures initially, it negatively a↵ects their intention to make use of the App when considering them complex [74]. On the other hand, Wang Egelandsdal, Amdam, Almli and Oostindjer [75] express that time and e↵ort may discourage users from using health Apps, thus arguing why PEOU can explain with more influence the ITU new technologies than PEU. A qualitative study by Tang, Abraham, Stamp and Greaves [76] revealed that the appearance and structure of a weight loss app’s interface could significantly influence the decision to use such technology. Sports providers should conduct better market research to better understand what fans want or expect in order to meet their expectations [77].

Analyzing the results of the relationships according to the age of the subjects in the sample, three di↵erent groups were established according to the average age of the sample or the age range with the highest percentage of representativeness. The first group consisted of studies with subjects under 25 years old [1,51–53,59,61], the second group consisted of participants aged between 25 and 30 years [42,43,56], a third group consisted of people over 30 years old [45,57,60] and one study was not grouped because the age of the sample ranged from 20 to 29 years [44]. Considering the di↵erent age groups and the relationships between the di↵erent factors, the results have shown that the group under 25 in the PU-ITU and ITU-AU relationships, this is, the studies with the larger young population, showed higher relationships of influence on the usefulness of the application for its use [1,51–53,59,60]. Young people were more likely to use fitness or sport Apps when they found them more functional, which meant that this intention had a great influence on the actual use of the app [1,53]. The studies in the group 25–30 and group over 30 found similar moderate relationships between PU-ITU, except for two studies in the group 25–30, one study observed a high influence between PU-ITU [59], and the study did not find PU to significantly influence UTIs [42].

On the other hand, the group 25–30 highlighted among the other two groups the relationships that have evaluated the PEOU factor, emphasizing the influence of this factor on ITU [42,43], while in the PEOU-PU relationship, the two studies of the group 25–30 that analyzed this relationship were the ones that had the greatest and least influence compared to the studies of the other groups [42,43]. The subjects in this intermediate group considered more the ease and intuition of the app to be able to perform it than the younger or older subjects. The group over 30 also found a moderate to high influence on the PEOU-ITU relationship [45,57]. However, the older group of studies were the ones that stood out in the PE influence on UTIs [45,60]; that is, an app that is fun will be more likely to be used. Finally, PE was shown to have a moderate influence on PEOU in the group under 25 [61] and group over 30 [45,57] and high influence in the group 25–30 [42].

Sustainability 2020, 12, 6641 15 of 24

Limitations and Future Research

Among the limitations of this review study could be started with the reduction of studies that are based on TAM and not the other existing theories or models. Likewise, a document search in the grey literature was not performed since the researchers marked a criterion that the studies should be collected in the di↵erent databases analyzed. Another aspect that could have conditioned the study was the focus only on studies that evaluated only fitness and physical activity Apps, discarding the studies that could evaluate the use of wearables or sports webs that would allow a broader analysis of existing studies on the intention to use in the sports context. Similarly, the years of research for the study could have been limited to the dates since the appearance of the Smartphone and the Apps, resulting in a large number of initial results that were not limited in time.

In addition, another of the limitations found is the lack of a common questionnaire that has been replicated in di↵erent studies and allows a better comparison of the di↵erent populations. This fact may be due to the fact that this line of research is very recent and very few studies have appeared that could lead to its replication and comparison of studies. In addition, all the studies choose the sample for convenience, whether the groups of university students or sports groups are easily accessible. The use of convenience sampling denotes a lack of application of more appropriate sampling that would allow the generalization of results to a larger universe. Some studies did refer to this limitation [44,78]. The age of the participants has been a widely reported limitation in the di↵erent studies [1,51,52,61,78] where the sample was mainly composed of young people born into the new digital society and whose beliefs may be very di↵erent from those of middle-aged adults who have not been raised in a digital society and who tend to have greater di�culties in starting to use the new technologies. Only three studies looked at a sample over 30 years old [45,57,60], while in the literature there is a study that evaluated the use of websites by people over 50 [79].

These di↵erences between age groups and therefore with digital gaps must also be approached from a sociodemographic point of view and with very di↵erent contexts. For example, Cho et al. [51] carried out a comparative study between the Korean and American populations, highlighting the existence of a limitation in this type of study due to the oversocialization of the individual characteristics of consumers and their contexts. Thus, some authors indicate that an individual’s socioeconomic status and educational level a↵ect their use of new technologies and Apps, since individuals with a higher level of education may have better digital skills [51]. In fact, they may have greater access to a larger number of technological devices, since not only Smartphones should be considered but also other devices such as tablets, laptops or even wearables [42]. This factor is important to take into account as currently, there are other technological devices that could use sports Apps (i.e., tablets, i-watch, etc.).

Similarly, the operating system could have an influence as not all Apps are available on the current large operating systems (Android and IOS). In addition, there are a multitude of di↵erent Apps that perform the same functions and many studies question the intention to use without determining what type of Apps the consumer uses. This fact could lead to a self-selection bias when filling out the questionnaire by the consumer considering the App that people use and not one common to the entire population [44].

With regard to other methodological aspects of the studies, it should be noted that all the studies found have had a cross-sectional approach, and no longitudinal study has been found to evaluate the evolution of the subject and his or her real intention to continue using said App. Chen and Lin [34] point out that there may be a discrepancy between the ITU of the App on the part of the consumer and the actual use that they make of this App afterwards. In general, this has been a relationship that some studies have considered [1,35,43]; however, as these are cross-sectional studies, it has not been possible to verify the change that they have had with respect to a lesser or greater use of the App. Likewise, not all studies have included other variables associated with TAM such as PE or PT which allow us to verify their influence on ITU in di↵erent groups and contexts.

Finally, this review has mainly focused on the search for quantitative studies, finding several qualitative studies but only two studies were found that used mixed methods to evaluate the ITU and

Sustainability 2020, 12, 6641 16 of 24

to be able to check what the user expresses in the questionnaire with the actual opinion about that App [75,80].

Future lines of research on the intention to use sports Apps or any other device should consider the inclusion of more variables of TAM in their model, as well as other variables specific to the Smartphone (social influence, attachment to the device, etc.), the sports context (motivation, commitment, participation) or variables traditionally linked to sports marketing such as satisfaction [44]. Therefore, common measuring instruments should be standardized to allow their application in di↵erent contexts. Similarly, the di↵erent age generations should be taken into account when evaluating and sampling in a population that includes consumers of di↵erent age ranges.

It would also be interesting to sample populations that do not refer to a single sport or discipline but rather have a variety of Apps from di↵erent sports or sports teams. Future work should also be carried out in a longitudinal way, being able to check whether the intention to use predisposed by the subject ends up being the real use of the application or not. Finally, it is interesting to contemplate studies that cover di↵erent social groups with cultural diversity analyzing not only the intention to use in the context itself but also in the individual characteristics of the consumer with di↵erent status and educational levels to evaluate the digital gap.

5. Conclusions

This systematic review responded to the need for a critical evaluation of existing research on the intentions of using sports Apps as this is an emerging field of research. The limited number of academic studies together with the deficiencies in some methodologies as can be seen in the risk analysis of research bias and the evidence found, has not allowed a more critical evaluation. These findings highlight the need for more rigorous and systematic research by researchers in the field, putting factors in common that allow a better evaluation of the context of the use of new technologies in the sports environment. At the same time, these findings have allowed the research team to identify a range of recommendations for sports organizations and researchers, which will help them to address future studies, and thus allow for a better growth and development of the evaluation of the intention to use Apps in sport.

Practical Implications

Sports organizations, sports marketing experts and new technology developers should make use of the considerations made in this study when developing or upgrading a sports App either for general sports information, a sports brand or exercise or health monitoring. Sports Apps have a great potential for the promotion and sponsorship of di↵erent products due to the potential use they have by consumers through advertising in these technologies. Suppliers of sports brands (retail, teams, etc.) should carry out an in-depth analysis of the fans by means of one of the tools evaluated that allow them to design the Apps according to the user’s expectations and to know what type of information or applicability they expect from it.

Another practical implication that is obtained is the possibility for sports organizations to develop Apps that have a gamification part that links the fan to consume the sports brand while he can enjoy certain benefits for using the organization’s App. They should also look for ways to engage the consumer with the App during the competition, i.e., generate exclusive content that can only be enjoyed by fans who have attended the live game, causing greater interest in the use of the App. Finally, to improve the quality of the Apps on Smartphones or other devices, industry and marketing professionals should examine all communication channels to ensure convenient access to the di↵erent content and services available.

Sustainability 2020, 12, 6641 17 of 24

In addition, the period of confinement caused by the COVID-19 pandemic, which has forced millions of people to stay at home, highlights the importance of this study to know the current situation on the topic of the intention to use fitness Apps by the population, especially for sports specialists and managers to know how to fit into the periods of new normality in which social distancing is forced. Although they have been in development for a long time, during the period of confinement there has been an increase in the o↵er of digital channels that help the guided practice of physical activity through the use of safe, simple and easy to implement programs and applications that cover activities related to cross-fit, yoga or dance activities for the general population or for the improvement of physical and mental well-being in older adults [18,81,82]. For experts in sports management or trainers, platforms such as Youtube or Zoom allow for individualized methods of physical exercise and real-time contact with the monitor, which allows for feedback to users regardless of where they are located [83]. Finally, Ammar et al. [84] suggest that future physical activity intervention to stay active during times of pandemic may be based on information and communication technologies, such as fitness Apps. Therefore, it is necessary to focus on a more in-depth study on the intention of the population to use these types of applications to promote active and healthy lifestyles.

Author Contributions: Conceptualization, S.A., J.G.-F., and M.G.-P.; methodology, S.A. and J.G.-F.; formal analysis, S.A.; investigation, S.A., J.G.-F. and I.V.; resources, I.V.; data curation, M.G.-P.; writing—original draft preparation, S.A., J.G.-F. and I.V.; writing—review and editing, S.A. and J.G.-F.; project administration, J.G.-F. and M.G.-P.; funding acquisition, J.G.-F. All authors have read and agreed to the published version of the manuscript.

Funding: This research was funded by University of Seville grant number 3840/0443/Study and design of technological consumer behavior in Spanish fitness centers, by Valgo Investment, S.L.U.

Conflicts of Interest: The authors declare no conflict of interest.

Sustainability 2020,12,6641

18 of24

A ppendix

A

Table A

1.A ssessm

entm ethodologicalquality

(part1).

ItemN Section

/Topic and

C hecklistItem

B eldad

& H

egner[43] B

yun et

al.[45] C

hen &

Lin [44]

C hiu

& C

ho [62]

C hiu

et al.[56]

C ho

et al.[59]

C ho

et al.[51]

C ho

& K

im [52]

D him

an etal.[58]

H a

et al.[42]

Title and

A bstract

1a Identification

ofthe type

ofstudy in

the title

1 1

1 1

1 1

1 1

1 1

1b Structured

sum m

ary ofobjective,m

ethods,results and

conclusions 0

1 1

1 1

1 1

1 1

0

Introduction B

ackground and

O bjectives

2a Scientific

background and

explanation ofrationale

1 1

1 1

1 1

1 1

1 1

2b Specific

objectives or

hypotheses 1

1 1

1 1

1 1

1 1

1

M ethods

Participants 3a

Eligibility criteria

for participants

1 1

0 0

0 0

1 0

0 1

3b Settings

and locations

w here

the data

w ere

collected 0

0 0

1 1

1 1

0 1

1 3c

A table

show ing

baseline dem

ographic characteristics

0 1

1 1

1 0

1 0

1 0

Sam ple

Size 4a

The sam

ple size

has been

determ ined

0 0

0 0

0 0

0 0

0 0

4b W

hen applicable,explanation

ofhow sam

ple size

w as

determ ined

0 0

0 0

0 0

0 0

0 0

Procedure

5 The

procedure has

su �

cientdetails to

allow replication,

including how

and w

hen they

w ere

actually adm

inistered 0

1 0

0 1

1 1

1 1

0

Instrum entorTools

6a C

om pletely

defined prespecified

prim ary

and secondary

outcom e

m easures,including

how and

w hen

they w

ere assessed

1 1

1 1

1 1

1 1

1 1

6b U

se ofvalidity

and reliability

tools. 1

1 1

1 1

1 1

1 1

1

Im plem

entation 7

W ho

m ade

each partofstudy

0 0

0 0

0 0

0 0

0 0

StatisticalM ethods

8a Statisticalm

ethods used

to analyze

the results

1 1

1 1

1 1

1 1

1 1

8b U

se ofM

ethods foradditionalanalyses

to objective

ofstudy 0

0 1

1 0

1 1

0 0

0

R esults

O utcom

es and

Estim ation

9 A

table orfigure

show ing

outputs ofanalysis

m ore

relevant ofstudy

1 1

1 1

1 1

1 1

1 1

Sustainability 2020,12,6641

19 of24

Table A

1.C ont.

ItemN Section

/Topic and

C hecklistItem

B eldad

& H

egner[43] B

yun et

al.[45] C

hen &

Lin [44]

C hiu

& C

ho [62]

C hiu

et al.[56]

C ho

et al.[59]

C ho

et al.[51]

C ho

& K

im [52]

D him

an etal.[58]

H a

et al.[42]

D iscussion

Interpretation

10 Interpretation

consistentw ith

results,balancing benefits

and harm

s and

considering other

relevantevidence 1

1 1

1 1

0 1

0 1

1

Lim itations

11 Study

lim itations,addressing

sources ofpotentialbias,

im precisions,etc.

1 1

1 1

1 1

1 0

1 1

PracticalIm plication

12 M

ain applicability

to results

ofstudy 1

1 1

1 1

1 1

0 1

1

O therInform

ation Funding

13 Sources

offunding and

other support,role

offunders 1

1 0

0 0

1 1

0 0

0

T O

TA L

12 15

13 14

14 14

17 9

14 12

Table A

2.A ssessm

entm ethodologicalquality

(part2).

ItemN Section

/Topic and

C hecklistItem

H uang

& R

en [60]

K im

et al.[1]

Lee et

al.[61] Lietal.

[63] M

oham m

adi& Isanejad

[57] N

dayizigam iye

etal.[54] W

eiet al.[55]

Yoo et

al.[53] Yuan

et al.[30]

Title and

A bstract

1a Identification

ofthe type

ofstudy in

the title

1 1

1 1

1 1

1 1

1

1b Structured

sum m

ary ofobjective,m

ethods,results and

conclusions 1

1 1

1 1

1 1

1 1

Introduction B

ackground and

O bjectives

2a Scientific

background and

explanation ofrationale

1 1

1 1

1 0

1 1

1 2b

Specific objectives

or hypotheses

1 1

1 1

0 1

1 1

1

M ethods

Participants 3a

Eligibility criteria

for participants

0 1

0 0

0 0

0 0

0 3b

Settings and

locations w

here the

data w

ere collected

1 1

0 1

0 1

0 0

1 3c

A table

show ing

baseline dem

ographic characteristics

1 1

1 1

1 0

1 1

0

Sam ple

Size 4a

The sam

ple size

has been

determ ined

0 0

0 0

0 0

0 0

0

4b W

hen applicable,explanation

ofhow sam

ple size

w as

determ ined

0 0

0 0

0 0

0 0

0

Sustainability 2020,12,6641

20 of24

Table A

2.C ont.

ItemN Section

/Topic and

C hecklistItem

H uang

& R

en [60]

K im

et al.[1]

Lee et

al.[61] Lietal.

[63] M

oham m

adi& Isanejad

[57] N

dayizigam iye

etal.[54] W

eiet al.[55]

Yoo et

al.[53] Yuan

et al.[30]

Procedure

5 The

procedure has

su �

cientdetails to

allow replication,

including how

and w

hen they

w ere

actually adm

inistered 0

1 0

0 0

0 0

0 0

Instrum entorTools

6a C

om pletely

defined prespecified

prim ary

and secondary

outcom e

m easures,including

how and

w hen

they w

ere assessed

1 1

1 1

1 1

1 1

0

6b U

se ofvalidity

and reliability

tools. 1

1 1

1 1

1 1

1 1

Im plem

entation 7

W ho

m ade

each partofstudy

0 0

0 0

0 0

0 0

0 StatisticalM

ethods 8a

Statisticalm ethods

used to

analyze the

results 1

1 1

1 1

1 1

1 1

8b U

se ofM

ethodsforadditionalanalysesto objective

ofstudy 0

0 1

0 0

0 0

0 1

R esults

O utcom

es and

Estim ation

9 A

table orfigure

show ing

outputs ofanalysis

m ore

relevant ofstudy

1 1

1 1

1 1

1 1

1

D iscussion

Interpretation

10 Interpretation

consistentw ith

results,balancing benefits

and harm

s and

considering other

relevantevidence 1

1 1

1 1

1 1

0 0

Lim itations

11 Study

lim itations,addressing

sources ofpotentialbias,

im precisions,etc.

1 1

0 1

0 0

1 0

1

PracticalIm plication

12 M

ain applicability

to results

ofstudy 1

1 1

1 1

0 1

0 0

O therInform

ation Funding

13 Sources

offunding and

other support,role

offunders 0

0 0

1 0

1 1

0 0

T O

TA L

13 15

12 14

10 10

13 9

10

Sustainability 2020, 12, 6641 21 of 24

References

1. Kim, Y.; Kim, S.; Rogol, E. The e↵ects of consumer innovativeness on sport team applications acceptance and usage. J. Sport Manag. 2017, 31, 241–255. [CrossRef]

2. Statista. Forecast Number of Mobile Users Worldwide from 2019 to 2023. Available online: https: //www.statista.com/statistics/218984/number-of-global-mobile-users-since-2010/ (accessed on 18 March 2020).

3. Cha↵ey, D. Mobile Marketing Statistics Compilation. Available online: https://www.smartinsights.com/mobi le-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ (accessed on 18 March 2020).

4. Fuchs, C. Information technology and sustainability in the information society. Int. J. Commun. 2017, 11, 2431–2461. Available online: https://ijoc.org/index.php/ijoc/article/view/6827 (accessed on 17 April 2020).

5. Luque-Ayala, A.; Marvin, S. Developing a critical understanding of smart urbanism? Urb. Stud. 2015, 52, 2105–2116. [CrossRef]

6. Vanolo, A. Smartmentality: The smart city as disciplinary strategy. Urb. Stud. 2014, 51, 883–898. [CrossRef] 7. Angelidou, M.; Psaltoglou, A.; Komninos, N.; Kakderi, C.; Tsarchopoulos, P.; Panori, A. Enhancing sustainable

urban development through smart city applications. J. Sci. Technol. Polic. Manag. 2018, 9, 146–169. [CrossRef] 8. Maaroof, A. Big Data and the 2030 Agenda for Sustainable Development. 2015. Available online:

https://www.unescap.org/events/call-participants-big-data-and-2030-agenda-sustainable-development-ac hieving-development (accessed on 2 August 2020).

9. United Nation. Transforming Our World: The 2030 Agenda for Sustainable Development. 2015. Available online: https://sustainabledevelopment.un.org/post2015/transformingourworld (accessed on 3 August 2020).

10. United Nations. Habitat III Issue Papers, 21—Smart Cities (V2.0). 2015. Available online: http://habitat3.org /wp-content/uploads/Habitat-III-Issue-Paper-21_Smart-Cities-2.0.pdf (accessed on 1 August 2020).

11. International Telecommunications Union. An Overview of Smart Sustainable Cities and the Role of Information and Communication Technologies. 2014. Available online: www.itu.int/en/ITU-T/focusgroups/s sc/Pages/default.aspx (accessed on 3 August 2020).

12. Vodafone Group. Unifying Communications. Annual Report. 2015. Available online: http://www.annualre ports.com/HostedData/AnnualReportArchive/v/LSE_VOD_2015.pdf (accessed on 22 March 2020).

13. Blair, I. Mobile App Download and Usage Statistics. 2019. Available online: https://buildfire.com/app-statis tics/ (accessed on 19 March 2020).

14. Statista. Number of Mobile App Downloads Worldwide in 2017, 2018 and 2022 (in Billions). Available online: https://www.statista.com/statistics/271644/worldwide-free-and-paid-mobile-app-store-downloads/ (accessed on 20 March 2020).

15. Statista. How Often do you Currently Make Use of Sports and Fitness Apps? Available online: https://www.stat ista.com/statistics/639567/sports-and-fitness-app-usage-frequency-in-us/ (accessed on 21 March 2020).

16. McKay, F.H.; Wright, A.; Shill, J.; Stephens, H.; Uccellini, M. Using Health and Well-Being Apps for Behavior Change: A Systematic Search and Rating of Apps. JMIR 2019, 7, e11926. [CrossRef]

17. World Health Organization. #HealthyAtHome—Physical Activity. Available online: https://www.who.int/ news-room/campaigns/connecting-the-world-to-combat-coronavirus/healthyathome/healthyathome—p hysical-activity (accessed on 15 May 2020).

18. Banskota, S.; Healy, M.; Goldberg, E.M. Smartphone Apps for Older Adults to Use While in Isolation During the COVID-19 Pandemic. West. J. Emerg. Med. 2020, 21, 514–525. [CrossRef]

19. IHRSA. Fitness Apps are Revolutionizing the Industry. Available online: https://www.ihrsa.org/improve-yo ur-club/fitness-apps-are-revolutionizing-the-industry/ (accessed on 27 April 2020).

20. McLean, G.; Osei-Frimpong, K.; Al-Nabhani, K.; Marriott, H. Examining consumer attitudes towards retailers’m-commerce mobile applications–An initial adoption vs. continuous use perspective. J. Bus. Res. 2020, 106, 139–157. [CrossRef]

21. Stocchi, L.; Michaelidou, N.; Micevski, M. Drivers and outcomes of branded mobile app usage intention. J. Prod. Brand Manag. 2019, 28, 28–49. [CrossRef]

22. Gao, T.; Rohm, A.J.; Sultan, F.; Huang, S. Antecedents of consumer attitudes toward mobile marketing: A comparative study of youth markets in the United States and China. Thund. Int. Bus. Rev. 2012, 54, 211–224. [CrossRef]

Sustainability 2020, 12, 6641 22 of 24

23. Maghnati, F.; Ling, K.C. Exploring the relationship between experiential value and usage attitude towards mobile apps among the smartphone users. Int. J. Bus. Manag. 2013, 8, 1–9. [CrossRef]

24. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 318–339. [CrossRef]

25. Fishbein, M.; Ajzen, I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research; Addison-Wesley: Reading, UK, 1975.

26. Ha, J.P.; Kang, S.J.; Ha, J. A conceptual framework for the adoption of smartphones in a sports context. Int. J. Sports Mark. Spons. 2015, 16, 2–19. [CrossRef]

27. Rivera, M.; Gregory, A.; Cobos, L. Mobile application for the timeshare industry: The influence of technology experience, usefulness, and attitude on behavioral intentions. J. Hosp. Tour. Technol. 2015, 6, 242–257. [CrossRef]

28. Lee, Y.; Hsieh, Y.; Hsu, C. Adding innovation di↵usion theory to the technology acceptance model: Supporting employees’ intentions to use e-learning systems. J. Educ. Technol. Soc. 2011, 14, 124–137. Available online: https://www.jstor.org/stable/jeductechsoci.14.4.124 (accessed on 20 April 2020).

29. Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. User acceptance of information technology: Toward a unified view. MIS Q. 2003, 27, 425–478. [CrossRef]

30. Yuan, S.; Ma, W.; Kanthawala, S.; Peng, W. Keep Using My Health Apps: Discover Users’ Perception of Health and Fitness Apps with the UTAUT2 Model. Telemed. E-Health 2015, 21, 735–741. [CrossRef]

31. Hur, Y.; Ko, Y.J.; Valacich, J. Motivation and concerns of online sport consumption. J. Sport Manag. 2007, 21, 521–539. [CrossRef]

32. Singh, N.; Sinha, N. How perceived trust mediates merchant’s intention to use a mobile wallet technology. J. Retail. Consum. Serv. 2020, 52, 101894. [CrossRef]

33. Chen, L.; Aklikokou, A.K. Determinants of E-government Adoption: Testing the Mediating E↵ects of Perceived Usefulness and Perceived Ease of Use. Int. J. Public Adm. 2020, 43, 850–865. [CrossRef]

34. Hossain, S.A.; Bao, Y.; Hasan, N.; Islam, M.F. Perception and prediction of intention to use online banking systems. Int. J. Res. Bus. Social Sci. 2020, 9, 112–116. [CrossRef]

35. Kaushik, A.K.; Mohan, G.; Kumar, V. Examining the Antecedents and Consequences of Customers’ Trust Toward Mobile Retail Apps in India. J. Internet Commer. 2020, 19, 1–31. [CrossRef]

36. Almaiah, M.A.; Al-Mulhem, A. Analysis of the essential factors a↵ecting of intention to use of mobile learning applications: A comparison between universities adopters and non-adopters. Educ. Inf. Technol. 2019, 24, 1433–1468. [CrossRef]

37. Carlson, J.; O’Cass, A. Optimizing the online cannel in professional sport to create trusting and loyal consumers: The role of the professional sports team brand and service quality. J. Sport Manag. 2012, 26, 463–478. [CrossRef]

38. Hur, Y.; Ko, Y.J.; Claussen, C.L. Acceptance of sport websites: A conceptual model. Int. J. Sport Mark. Spons. 2011, 12, 209–224. [CrossRef]

39. Hur, Y.; Ko, Y.J.; Claussen, C.L. Determinants of using sports web portals: An empirical examination of the sport website acceptance model. Int. J. Sport Mark. Spons. 2012, 13, 6–25. [CrossRef]

40. Cilletti, D.; Lanasa, J.; Ramos, D.; Luchs, R.; Lou, J. Sustainability Communication in North American Professional Sport Leagues: Insights from web-site self-presentations. Int. J. Sport Commun. 2010, 3, 64–69. [CrossRef]

41. Kim, T.; Chiu, W. Consumer acceptance of sports wearable technology: The role of technology readiness. Int. J. Sport Mark. Spons. 2019, 20, 109–126. [CrossRef]

42. Ha, J.P.; Kang, S.J.; Kim, Y. Sport fans in a “smart sport” (SS) age: Drivers of smartphone use for sport consumption. Int. J. Sport Mark. Spons. 2017, 18, 281–297. [CrossRef]

43. Beldad, A.D.; Hegner, S.M. Expanding the Technology Acceptance Model with the Inclusion of Trust, Social Influence, and Health Valuation to Determine the Predictors of German Users’ Willingness to Continue using a Fitness App: A Structural Equation Modeling Approach. Int. J. Hum. Comput. Interact. 2018, 34, 882–893. [CrossRef]

44. Chen, M.F.; Lin, N.P. Incorporation of health consciousness into the technology readiness and acceptance model to predict app download and usage intentions. Internet Res. 2018, 28, 351–373. [CrossRef]

45. Byun, H.; Chiu, W.; Bae, J.S. Exploring the adoption of sports brand apps: An application of the modified technology acceptance model. Int. J. Asian Bus. Inf. Manag. 2018, 9, 52–65. [CrossRef]

Sustainability 2020, 12, 6641 23 of 24

46. Song, J.; Kim, J.; Cho, K. Understanding users’ continuance intentions to use smart-connected sports products. Sport Manag. Rev. 2018, 21, 477–490. [CrossRef]

47. Rohm, A.J.; Gao, T.T.; Sultan, F.; Pagani, M. Brand in the hand: A cross-market investigation of consumer acceptance of mobile marketing. Bus. Horiz. 2012, 55, 485–493. [CrossRef]

48. Schulz, K.F.; Altman, D.G.; Moher, D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMC Med. 2010, 340, c332. [CrossRef]

49. Moher, D.; Liberati, A.; Tetzla↵, J.; Altman, D.G. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [CrossRef]

50. Lane, D. Fisher r-to-z Calculator. Available online: http://onlinestatbook.com/calculators/ (accessed on 17 May 2020).

51. Cho, J.; Lee, H.E.; Quinlan, M. Cross-National Comparisons of College Students’ Attitudes toward Diet/Fitness Apps on Smartphones. J. Am. Coll. Health 2016, 65, 437–439. [CrossRef]

52. Cho, J.; Kim, S.J. Factors of Leading the Adoption of Diet/Exercise Apps on Smartphones: Application of Channel Expansion Theory. J. Korea Soc. Internet Inf. 2015, 16, 101–108. [CrossRef]

53. Yoo, D.H.; Ko, D.S.; Yeo, I.S. E↵ect of user’s trust in usefulness, attitude and intention for mobile sports content services. J. Phys. Educ. Sport 2017, 17, 92–96. [CrossRef]

54. Ndayizigamiye, P.; Kante, M.; Shingwenyana, S. An adoption model of mHealth applications that promote physical activity. Cogent Psychol. 2020, 7, 1764703. [CrossRef]

55. Wei, J.; Vinnikova, A.; Lu, L.; Xu, J. Understanding and Predicting the Adoption of Fitness Mobile Apps: Evidence from China. Health Commun. 2020, 1–12. [CrossRef] [PubMed]

56. Chiu, W.; Cho, H.; Chi, C.G. Consumers’ continuance intention to use fitness and health apps: An integration of the expectation–confirmation model and investment model. Inf. Technol. People 2020. [CrossRef]

57. Mohammadi, S.; Isanejad, O. Presentation of the Extended Technology Acceptance Model in Sports Organizations. Ann. Appl. Sport Sci. 2018, 6, 75–86. [CrossRef]

58. Dhiman, N.; Arora, N.; Dogra, N.; Gupta, A. Consumer adoption of smartphone fitness apps: An extended UTAUT2 perspective. J. Indian Bus. Res. 2019, 12, 363–388. [CrossRef]

59. Cho, J.; Lee, H.E.; Kim, S.J.; Park, D. E↵ects of body image on college students’ attitudes toward diet/fitness apps on smartphones. Cyberpsychol. Behav. Soc. Netw. 2015, 18, 41–45. [CrossRef]

60. Huang, G.; Ren, Y. Linking technological functions of fitness mobile apps with continuance usage among Chinese users: Moderating role of exercise self-e�cacy. Comput. Human Behav. 2020, 103, 151–160. [CrossRef]

61. Chiu, W.; Cho, H. The role of technology readiness in individuals’ intention to use health and fitness applications: A comparison between users and non-users. Asia Pacific J. Mark. Logist. 2020. [CrossRef]

62. Lee, S.; Kim, S.; Wang, S. Motivation factors influencing intention of mobile sports apps use by applying the unified theory of acceptance and use of technology (UTAUT). Int. J. Appl. Sports Sci. 2017, 29, 115–127. [CrossRef]

63. Li, J.; Liu, X.; Ma, L.; Zhang, W. Users’ intention to continue using social fitness-tracking apps: Expectation confirmation theory and social comparison theory perspective. Inform. Health Soc. Care 2019, 44, 298–312. [CrossRef]

64. Walter, T. Worldwide survey of fitness trends for 2017. ACSM’s Health Fit. J. 2016, 20, 8–17. [CrossRef] 65. Hur, W.M.; Kim, H.; Kim, W.M. The moderating roles of gender and age in tablet computer adoption.

Cyberpsychol. Behav. Soc. Netw. 2014, 17, 33–39. [CrossRef] [PubMed] 66. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. User acceptance of computer technology: A comparison of two

theoretical models. Manag. Sci. 1989, 35, 982–1003. [CrossRef] 67. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. Extrinsic and Intrinsic Motivation to Use Computers in the Workplace.

J. Appl. Soc. Psychol. 1992, 22, 1111–1132. [CrossRef] 68. Hong, S.; Thong, J.Y.L.; Tam, K.Y. Understanding continued information technology usage behavior:

A comparison of three models in the context of mobile internet. Decis. Support Syst. 2006, 42, 1819–1834. [CrossRef]

69. Moon, J.W.; Kim, Y.G. Extending the TAM for a World-Wide-Web context. Inf. Manag. 2001, 38, 217–230. [CrossRef]

70. Venkatesh, V.; Davis, F.D. A theoretical extension of the technological acceptance model: Four longitudinal field studies. Manag. Sci. 2000, 46, 186–204. [CrossRef]

Sustainability 2020, 12, 6641 24 of 24

71. Sun, H.; Zhang, P. Causal relationships between perceived enjoyment and perceived ease of use: An alternative approach. J. Assoc. Inf. Syst. 2006, 7, 618–645. [CrossRef]

72. Klenk, S.; Reifegerste, D.; Renatus, R. Gender di↵erences in gratifications from fitness app use and implications for health interventions. Mob. Media Commun. 2017, 5, 178–193. [CrossRef]

73. Baptista, G.; Baptista, G.; Oliveira, T.; Oliveira, T. Why so serious? Gamification impact in the acceptance of mobile banking services. Internet Res. 2017, 27, 118–139. [CrossRef]

74. Gowin, M.; Cheney, M.; Gwin, S.; Wann, T.F. Health and fitness app use in college students: A qualitative study. Am. J. Health Educ. 2015, 46, 223–230. [CrossRef]

75. Wang, Q.; Egelandsdal, B.; Amdam, G.V.; Almli, V.L.; Oostindjer, M. Diet and physical activity apps: Perceived e↵ectiveness by app users. JMIR mHealth uHealth 2016, 4, e33–e47. [CrossRef] [PubMed]

76. Tang, J.; Abraham, C.; Stamp, E.; Greaves, C. How can weight-loss app designers’ best engage and support users? A qualitative investigation. Br. J. Health Psychol. 2015, 20, 151–171. [CrossRef] [PubMed]

77. Yoon, C.; Jeong, C.; Rolland, E. Understanding individual adoption of mobile instant messaging: A multiple perspectives approach. Inf. Technol. Manag. 2015, 16, 139–151. [CrossRef]

78. Cheung, D.S.T.; Or, C.K.L.; So, M.K.P.; Tiwari, A. Usability Testing of a Smartphone Application for Delivering Qigong Training. J. Med. Syst. 2018, 42, 191–199. [CrossRef]

79. Tseng, K.C.; Hsu, C.; Chung, Y. Acceptance of information technology and the internet by people aged over fifty in Taiwan. Soc. Behav. Per. Int. J. 2012, 40, 613–622. [CrossRef]

80. Kang, S.J.; Ha, J.P.; Hambrick, M.E. A mixed-method approach to exploring the motives of sport-related mobile applications among college students. J. Sport Manag. 2015, 29, 272–290. [CrossRef]

81. Chen, P.; Mao, L.; Nassis, G.P.; Harmer, P.; Ainsworth, B.E.; Li, F. Coronavirus disease (COVID-19): The need to maintain regular physical activity while taking precautions. J. Sport Health Sci. 2020, 9, 103–104. [CrossRef]

82. Nyenhuis, S.M.; Greiwe, J.; Zeiger, J.S.; Nanda, A.; Cooke, A. Exercise and Fitness in the age of social distancing during the COVID-19 Pandemic. J. Allerg. Clin. Immun. 2020, 8, 2152–2155. [CrossRef]

83. Ng, K. Adapted physical activity through COVID-19. Eur. J. Adap. Phys. Act. 2020, 13, 1. [CrossRef] 84. Ammar, A.; Brach, M.; Trabelsi, K.; Chtourou, H.; Boukhris, O.; Masmoudi, L.; Bouaziz, B.; Bentlage, E.;

How, D.; Ahmed, M.; et al. On Behalf of the ECLB-COVID19 Consortium. E↵ects of COVID-19 Home Confinement on Eating Behaviour and Physical Activity: Results of the ECLB-COVID19 International Online Survey. Nutrients 2020, 12, 1583. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

ORIGINAL ARTICLE

A systematic review of the application of interactive virtual reality to sport

David L. Neumann1,2 • Robyn L. Moffitt1,2 • Patrick R. Thomas2 •

Kylie Loveday1 • David P. Watling1 • Chantal L. Lombard1 • Simona Antonova1 •

Michael A. Tremeer1

Received: 26 April 2016 / Accepted: 10 July 2017 / Published online: 19 July 2017 ! Springer-Verlag London Ltd. 2017

Abstract Virtual reality (VR) technology is being increasingly used by athletes, coaches, and other sport-re-

lated professionals. The present systematic review aimed to

document research on the application of VR to sport to better understand the outcomes that have emerged in this

work. Research literature databases were searched, and the

results screened to identify articles reporting applications of interactive VR to sport with healthy human participants.

Twenty articles were identified and coded to document the

study aims, research designs, participant characteristics, sport types, VR technology, measures, and key findings.

From the review, it was shown that interactive VR appli-

cations have enhanced a range of performance, physio- logical, and psychological outcomes. The specific effects

have been influenced by factors related to the athlete and

the VR system, which comprise athlete factors, VR envi- ronment factors, task factors, and the non-VR environment

factors. Important variables include the presence of others

in the virtual environment, competitiveness, task auton- omy, immersion, attentional focus, and feedback. The

majority of research has been conducted on endurance sports, such as running, cycling, and rowing, and more

research is required to examine the use of interactive VR in

skill-based sports. Additional directions for future research and reporting standards for researchers are suggested.

Keywords Virtual reality ! Sport ! Exercise ! Systematic review

1 Introduction

The application of computer-based technology to sport is an area of intense interest. Such technologies include

computerised modelling, data acquisition and analysis,

mobile computers, and information technology networks (Baca et al. 2009). Virtual reality (VR) is another tech-

nology, and it was first applied to sport research in the

1990s, although there has been a resurgence of interest in recent years. VR refers to a computer-simulated environ-

ment that aims to induce a sense of being mentally or

physically present in another place (Baños et al. 2000; Sherman and Craig 2002). An important feature of VR is

that the individual can interact with the environment. In the

context of sport, interaction might occur through an exer- tion interface (Mueller et al. 2007). For example, physical

effort on a machine such as an ergometer can be related to

the speed of movement through a virtual race course. Motion capture video systems, infrared beams, and wear-

able sensors are other approaches that can be used to translate physical actions into virtual sport performance.

The key elements that define VR applications to sport

are the use of computer-generated sport-relevant content and a means for the athlete to interact with the virtual

environment. When defined in this way, the application of

VR to sport has a number of strengths. As noted by Hoffman et al. (2014), the VR environment can be con-

trolled and manipulated in specific and reproducible ways.

Hoffman et al. used these characteristics to train partici- pants to use a rowing race pacing strategy. VR can also be

used for assessment, to gain feedback on performance, and

& David L. Neumann [email protected]

1 School of Appled Psychology, Griffith University, Gold Coast, Queensland 4222, Australia

2 Menzies Health Institute Queensland, Gold Coast, Queensland 4222, Australia

123

Virtual Reality (2018) 22:183–198

https://doi.org/10.1007/s10055-017-0320-5

to practice specific skills. The VR environment does not

need to be limited to a single person. Other individuals may be present such as a coach, teammate, or competitor even if

they are physically located in another place. The ability to

connect with individuals via the Internet allows for inter- action without the need for travel. Finally, the increasing

availability of commercially produced software or full VR

systems avoids the need for specialised technical expertise and allows VR to be used in local gyms and at home.

The present study aimed to provide a systematic review of research on VR applications to sport. The PsycINFO,

SPORTDiscus, Scopus, Google Scholar, and Cochrane

Library databases were first searched for the existence of similar reviews. The search yielded systematic reviews on

VR in physical rehabilitation (e.g. Laver et al. 2015), VR in

psychological interventions (e.g. Meyerbröker and Emmelkamp 2010), and the use of exergames or active

videogames (e.g. Guy et al. 2011; Larsen et al. 2013; Peng

et al. 2013). The search helped to minimise overlap with existing reviews. Accordingly, the present review focused

on VR applications to sport and sport-related exercise with

healthy individuals. Studies were included if they were based on recognised sports even if those sports are used as

a component of physical conditioning or fitness programs

(e.g. cycling, running, rowing). As a result, this review focused on sport-based tasks as distinct from research with

interactive videogame systems that promote physical

activity through gameplay (i.e. exergames). The broad question examined in the present review was:

What is known about the application of VR to sport? In

particular, the review aimed to provide a definition of VR when used for sports. A further aim was to document the

aims, methods, and the broad findings from the research

conducted to date. Past research may be interpreted within the context of existing theories in sport and exercise, but of

particular focus in the present review were those factors

that are unique to VR applications to sport. The review also aimed to identify the gaps in the research to date and

develop recommended reporting standards for researchers

who apply VR to sport.

2 Literature review method

The literature search and selection method followed the

Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines (Liberati et al. 2009)

and the use of inclusion and exclusion rules described by

Meline (2006). Initially, the SPORTDiscus and PsycINFO databases were searched. The PsycINFO database includes

sport and exercise psychology journals, in addition to the

ACM Transactions on Applied Perception, the ACM Transactions on Computer–Human Interaction, and the

IEEE Transactions on Professional Communication. The

search was conducted using the terms: (sport* OR exercis* OR fitness OR physical train* or physical activit*) AND

(virtual realit* OR virtual environment* OR virtual world*

OR virtual system* OR virtual partner*). The search was limited to articles published from 1990 and up to the date

of the search (February, 2016) and included articles that

were in press. In addition, to identify any missed articles due to the inconsistent use of terms (e.g. virtual reality

versus virtual competitor) the reference lists of the articles selected for final inclusion from the database search were

examined. An examination was also made of the citations

of these articles, as collated from the Scopus database. The database search yielded 263 articles from the Psy-

chINFO database and 377 articles from the SPORTDiscus

database for a total of 640 articles. This reduced to 620 articles following removal of duplicates. A search of the

reference lists and citations yielded a further 66 unique

articles. Articles were screened for exclusion or inclusion by two individuals in a two-step process: title and abstract

(Step 1) and the full article (Step 2).1 The following

exclusion criteria were used: date (published before 1990), language (not published in English language), source (a

dissertation, thesis, abstract only, magazine article, or not a

peer-reviewed source), study type (a review, meta-analysis, commentary, letter to the editor, editorial report, or other

non-empirical article), no VR was used (a computer-gen-

erated environment was not used or there was no interac- tivity with the environment), population (the sample did

not include healthy human participants), task (the methods

did not include participation in a sport or a physical exer- cise that used equipment related to a sport or sports train-

ing), game (the task was based wholly on an exergame/

active videogame), rehabilitation (the purpose of the task was to rehabilitate those with physical injury), and measure

(performance, physiological, or psychological outcomes

were not the primary measures). Following the screening and selection process, 20 arti-

cles were included for full review. Of these articles, 18

were published in journals with journal citation metrics reported by the Web of Science database. The mean impact

factor (based on the most recent year) was 2.21 (range

0.06–4.47, SD = 1.21) indicating that the journals were largely of good quality although with some exceptions.

Consistent with this interpretation, the journal rankings

varied evenly across the full spectrum of Q1 (n = 5), Q2 (n = 5), Q3 (n = 4), and Q4 (n = 4). The articles were

coded by four authors and coding decisions were cross-

1 Cohen’s kappa for the decisions to exclude or include based on title and abstract (Step 1) was j = 0.64 and based on review of full article (Step 2) was j = 0.69, both of which fall within the guidelines for substantial agreement. Full agreement was reached at each step following discussion.

184 Virtual Reality (2018) 22:183–198

123

checked. Articles were coded for characteristics related to

the study (aims, type, location, conditions/groups, outcome measures, key findings), participants (sample size, age,

experience with sport), virtual reality technology (task

type, system, display features, point of view, others in the environment, immersion/presence measures), and sport

task (type).

3 Defining virtual reality in sport

VR when applied to sport may be defined as instances

when individuals are engaged in a sport that is represented in a computer-simulated environment which aims to induce

a sense of being mentally or physically present and enables

interactivity with the environment. This definition high- lights the computer-simulated nature and interactivity of

the virtual environment, which are key element of more

general definitions of VR (e.g. Baños et al. 2000; Sherman and Craig 2002). It also aims to highlight the application of

VR to sport from the perspective of the user (athlete).

Realistic responses to virtual environments are suggested to occur when the system induces a sense of presence and

the perception that the events are actually occurring (Slater

2009). In this respect, it is important that VR uses a computer-generated environment because this is a key

feature that allows for interactivity and the perception of

presence (Baños et al. 2000; Sherman and Craig 2002). In other words, the virtual environment or elements within it

will move or change in response to the actions of the

athlete. However, the method by which the virtual envi- ronment is presented to the athlete should not be specified

in the definition because it might impose technological

limitations to the application of VR to sport (see Steuer 1992).

In many applications outside of sport, the virtual envi-

ronment is displayed using a computer automatic virtual environment (CAVE) or head-mounted display (HMD).

The CAVE is composed of a large cube made up of display

screens that the user physically enters to become sur- rounded by the virtual environment. A HMD is a wearable

device that covers the eyes and thus removes vision of the

outside world. It has one or more small screens on which the virtual world is viewed in stereovision with a wide field

of view. The HMD is combined with head tracking to allow

the user to view areas of the virtual environment that are outside of the immediate field of view by turning their

head. Being a smaller, more portable, and a more afford-

able system, the HMD is more popular than the CAVE, although both may be regarded as sharing the same key

features of an immersive system (Slater 2009).

However, the potential applications for using CAVE and HMD systems can be limited for some types of sports.

A HMD may be impractical or potentially dangerous for

some sports. For example, running a race on a treadmill using a HMD can be hazardous because vision of the

moving treadmill is removed. The head movements and

sweating of the athlete can also make the HMD uncom- fortable to wear. Indeed, in no studies identified in this

review was a HMD system used despite researchers con-

sistently using the term virtual reality to describe their approach. The most common approach was a two-dimen-

sional depiction of the virtual environment using a com- puter screen or a projector. A computer screen or projector

has the advantages of ease of use and practicality with sport

but may induce less presence than a HMD or CAVE sys- tem. Further research is required to determine whether

there is significant difference in presence when a computer

screen or projector is used. Several instances can be identified in which researchers

used methodology that approximated the proposed defini-

tion of VR applications to sport. For example, some researchers have used a visual display that shows a video of

a real environment (e.g. Plante et al. 2006). Feltz et al.

(2011) conducted a series of studies that investigated the Köhler motivation gain effect with a plank exercise task.

These studies showed the participant via a video (i.e. not a

computer-generated avatar) and included a second indi- vidual shown on a second visual display without any

interaction. Videos of real environments and people may

have potential for VR applications to sport, but they must include elements of interactivity to fulfil the proposed

definition of VR. Similarly, other researchers have used

computer-generated environments to examine baseball batting (Ranganathan and Carlton 2007), handball goal-

keeping (Vignais et al. 2015), and soccer goalkeeping

(Stinson and Bowman 2014), but these did not allow for any interactivity with the environment and were not

included in the review. Thus, the present review was

focussed more specifically on interactive VR applications to sport. In some cases, it was also found that researchers

used a non-animated avatar against a blank screen (e.g.

Briki et al. 2013), but these do not meet the proposed definition because the methods did not simulate a real

environment.

Another important consideration for interactive VR applications to sport is the distinction between sport,

exercise, and exergaming. Sport may be defined as an

activity that requires motor skill and/or hand-eye coordi- nation combined with physical exertion and includes rules

and elements of competition (Australian Bureau of Statis-

tics 2008). Exercise, used synonymously with physical exercise, is a structured activity that may include repetitive

elements that is performed to maintain or improve physical

fitness (Australian Bureau of Statistics 2008). Exergame/ active videogame is a videogame played on commercial

Virtual Reality (2018) 22:183–198 185

123

game console systems (e.g. Xbox, Wii, PlayStation) that

combines gameplay with physical movements that are more than sedentary behaviour (Kim et al. 2014). Exercise

and exergames together represent a more general case of

enhancing physical activity and may not necessarily be based on a sport.

Exercises or exergames that are not based on a sport

clearly do not represent instances of VR applications to sport even if they incorporate a virtual environment.

However, investigators have used sport-related computer games, particularly those that run on a games console, in

research. Console games based on sports have been used to

examine skill acquisition and transfer in children (Rey- nolds et al. 2014) and adults (Tirp et al. 2015). However,

these applications lacked an appropriate exertion interface

(e.g. participants ran on the spot to simulate running in the game) or essential sporting equipment (e.g. no darts were

used in a dart game), and these aspects can make the task

substantially different to perform the sport in real life. VR has also been applied to exercise and improving physical

fitness. In several studies, researchers have used sport-re-

lated tasks such as cycling, running, and rowing (e.g. Murray et al. 2016). These applications have relevance to

sport performance particularly because many of these

studies have introduced elements of competition or pres- sure to meet team goals.

4 A conceptual framework for the application of virtual reality to sport

The application of VR to sport has taken many forms, with

various types of sport tasks, VR technologies, and types of

athletes used in the research. Some researchers have examined questions relating to the use of VR technology

itself, such as comparing outcomes when using VR and not

using VR (e.g. Annesi and Mazas 1997; Legrand et al. 2011; Mestre et al. 2011; Plante et al. 2003a), the effects of

immersion in the virtual environment (Ijsselsteijn et al.

2004; Vogt et al. 2015), and differences between computer- controlled and real virtual competitors (Snyder et al. 2012).

In contrast, other researchers have used VR technology as

part of a methodology to answer more general questions about factors related to sport performance. For example,

Oliveira et al. (2015) used a virtual partner as a means to

compare the effects of self-selected and externally imposed exercise intensity.

We developed a broad conceptual model that sum-

marises and provides a framework to interpret the research conducted to date. As shown in Fig. 1, the VR system

results in outcomes that occur concurrently or following

engagement in the VR sport task. The VR system is composed of four components. These are the VR

environment, the sport task, the athlete, and the non-VR

environment. Research on VR applications to sport have largely focussed on only the first three of these compo-

nents. The VR environment is the unique component for

VR applications to sport and is the focus of most research. The second component, the sport task used, will differ

according to the application and can vary between endur-

ance-type sports or skill-based sports. The third component relates to characteristics of the athlete, such as skill level

and competitiveness. The characteristics of the athlete may act independently or they may interact with other elements

of the VR system to influence outcomes. The fourth com-

ponent encompasses those aspects of the real-world envi- ronment in which the athlete completes the task. Ambient

temperature, humidity, and time of day are among the

relevant factors that can be present and influence outcomes. Finally, all four elements of the VR system will produce

outcomes that emerge on an ongoing basis when per-

forming the sport task (concurrent outcomes) or they may emerge at a later time (posttask outcomes). The posttask

outcomes may be short term or long term.

The four components of the VR system share elements in common with other models applied to sport and exercise

psychology. For example, Tenenbaum and Hutchinson

(2007) proposed that perceived effort and effort tolerance are determined by the individual (e.g. dispositions, task

familiarity, demographic characteristics), the task (e.g.

intensity, duration), and the environmental conditions (e.g. social, physical features) that are present in a given situa-

tion. These conditions are analogous to the three non-VR

components of the VR system as presented in Fig. 1. Such a similarity is to be expected because VR aims to simulate

a real environment. However, research on VR applications

to sport have not yet examined the effects of the real (non- VR) environment on performance. Instead, attention has

been directed towards variables related to the virtual

environment, such as immersion, presence, and interac- tivity with virtual others. Research supporting the con-

ceptual framework depicted in Fig. 1 is presented below.

5 The virtual reality system

5.1 Virtual reality environment and task factors

The first two components, the VR environment and sport task, may be considered together because they can be

closely linked. For example, a rower may complete a time

trial using a rowing ergometer. However, the ergometer is merely the exertion interface. It is transformed into a vir-

tual boat such that pulls on the ergometer handle are

depicted as movements of the virtual oars through the water. Increasing exertion on the task (e.g. rowing at a

186 Virtual Reality (2018) 22:183–198

123

higher intensity) will be reflected in changes in the virtual

environment (e.g. faster movement through the water and passing scenery). Thus, performance and other factors

related to the task will influence the virtual environment

and this relationship can be reciprocal. Research has shown that several characteristics of the VR

environment and the task influence outcomes. A summary of

the methodological approaches used to create the VR envi- ronment and task is provided in Table 1. As can be seen, the

sport tasks used most often have been cycling and running, but rowing, weightlifting, and golf have also been examined.

Cycling, running, and rowing are sports that contain ele-

ments of endurance and persistence. These sports are also relatively easy to translate into a virtual environment. The

exertion interface of the treadmill or ergometer can readily

monitor information with regard to the speed and other performance elements (e.g. cadence) and translate this

information into virtual movements. Interactivity is further

enhanced by including directional controls although few VR systems have been used which have this capability.

The VR software and display equipment used in

research has varied from commercially available products to those that are custom made. The virtual environment is

typically displayed on computer screens or projected

against a wall. A larger display or the inclusion of more multimodal elements of the environment will increase the

sense of immersion in the virtual world (Vogt et al. 2015)

and this can influence performance. Using a more immer- sive virtual environment during a cycling task (i.e. showing

the track from the point of view of the rider versus from a

birds eye view) has increased motivation and the speed of cycling in participants (Ijsselsteijn et al. 2004). Using a

virtual running task, over a third of participants have reported that the immersion induced by the VR environ-

ment is an important motivating feature (Nunes et al.

2014). There might be a dose-dependent relationship between the level of immersion induced by the VR system

and the magnitude of the resulting outcomes.

The presence of others in the virtual environment has also emerged as an important feature of the VR environ-

ment. Indeed, the presence of others may be even more

important than the capability of the VR system to induce feelings of immersion or the presence. In a survey study

examining golf play in a virtual environment, Lee et al.

(2012) distinguished between two types of presence: telepresence or the feeling of being physical immersed in

VR Display ■ screen ■ HMD ■ CAVE

Exertion interface

Athlete

Location

Social

CONCURRENT OUTCOMES

POSTTASK OUTCOMES

Short-term Long-term

Physiological

Psychological

Performance

Sport Pr

es en

ce o

f o th

er s

VR SYSTEM

Fig. 1 A model of interactive virtual reality (VR) in sport and sport-related exercise showing the relationship between components of the VR system, current outcomes, and posttask outcomes

Virtual Reality (2018) 22:183–198 187

123

the virtual environment and social presence or the feeling

of being with and communicating with others in the virtual environment. Social presence was shown to play a more

important role in perceived enjoyment, perceived value,

and behavioural intentions than telepresence. Further, unlike social presence, telepresence did not significantly

predict any of these outcomes.

The presence of others has also influenced motivation and performance for aerobic sport tasks. Using a running

task, Nunes et al. (2014) reported that participants pre-

ferred to run in the presence of virtual others than to run on the virtual course alone (Nunes et al. 2014). Irwin et al.

(2012) examined the Köhler motivation gain effect while

participants cycled in a virtual environment. Participants cycled at an intensity of 65% of heart rate reserve for as

long as they felt comfortable. Different groups of partici-

pants completed trials while cycling in the virtual envi- ronment alone or at the same time as another person (a

Table 1 Characteristics of the task and virtual reality system of studies investigating virtual reality in sport

Authors Years Task Sport equipment VR technology VR display

Point of view

Others in environment

Anderson- Hanley et al.

2011 Cycling Recumbent stationary bicycle

Netathlon riding software Laptop screen

Not specified

Some conditions

Anderson- Hanley et al.

2012 Cycling Recumbent stationary bicycle

Netathlon riding software Laptop screen

Not specified

Yes

Anderson- Hanley et al.

2014 Cycling Recumbent stationary bicycle

Netathlon riding software Laptop screen

Not specified

Yes

Annesi and Mazas

1997 Cycling Stationary recumbent bicycle

Tectrix VR bike Screen Not specified

Yes

Baños et al. 2016 Walking Treadmill Commercial VR exergaming platform) Projected Third Not specified

Chen et al. 2015 Weightlifting Dumbbells Cave automatic virtual environment (CAVE)

Projected Not specified

No

Hoffman et al.

2014 Rowing Indoor rowing ergometer

Not specified Screen First Not specified

Ijsselsteijn et al.

2004 Cycling Stationary racing bicycle

Tacx T19000i-magic’ VR trainer Projected First and third

Yes

Irwin et al. 2012 Cycling Stationary bicycle Expresso fitness bike system Screen Not specified

Some conditions

Lee et al. 2012 Golf Golf ball and clubs Golfzon managed and operated virtual golf simulator

Projected First Not specified

Legrand et al.

2011 Running or cycling

Treadmill and regular bicycle ergometer

Tacx I-magic fortius Projected Not specified

No

Mestre et al. 2011 Cycling Stationary bicycle Tacx VR trainer Screen Third Yes

Murray et al.

2016 Rowing Indoor rowing ergometer

Netathlon 2 XF software Projected Third Yes

Nunes et al. 2014 Running Treadmill Running wheel Screen Third Some conditions

Oliviera et al.

2015 Cycling Cycle ergometer CompuTrainer 3D software Screen Not specified

Yes

Plante et al. 2003a Cycling Stationary bicycle Trek extreme mountain biking and cycle Fx ITS-1 with ultra-coach VR lite

Screen Third Yes

Plante et al. 2003b Cycling Stationary bicycle Trek extreme mountain biking and cycle Fx ITS-1

Screen Third Yes

Sigrist et al. 2015 Rowing Indoor rowing ergometer

Cave automatic virtual environment (CAVE)

Projected Not specified

No

Snyder et al. 2012 Cycling Recumbent stationary bicycle

Cybercycle expresso S3R Screen Third Yes

Vogt et al. 2015 Cycling Cycle ergometer Custom made Projected First No

VR virtual reality

188 Virtual Reality (2018) 22:183–198

123

confederate) who the participant was informed had per-

formed moderately better than they did in a baseline trial. Cycling with the other person was either in a conjunctive

situation (a ‘‘team score’’ would be based on the rider who

quit the task first) or a coactive situation (no team part- nership). Task persistence was higher in the coactive sit-

uation than when cycling alone. Moreover, a further

enhancement of persistence was observed in the conjunc- tive situation, suggesting motivational gains when per-

forming a VR-based sport in a team situation. In the study by Irwin et al. (2012), the confederate was

shown via a video loop on another screen and not in the VR

environment. Murray et al. (2016) also examined the Köhler motivation gain effect in which the teammate was

present as a virtual partner in the virtual environment.

Female participants novice to rowing completed a rowing trial in the presence of a virtual teammate in a conjunctive

situation (the shortest distance rowed over a 9-min trial

would count as the team score) or in the VR environment alone. Prior to the trial, participants were informed that the

teammate had rowed 40% longer than them in an initial

baseline row. A Köhler motivation gain effect was found in that participants rowed further and had a higher heart rate

in the presence of a teammate than when rowing in the VR

environment alone. Moreover, the conditions did not differ in felt arousal, positive feelings, or ratings of perceived

exertion. The latter finding suggests that performance

improvements can be induced by a virtual partner in the absence of negative psychological costs.

The presence of others in a virtual environment can be

used to more directly induce a pressure to perform in a competitive situation. Using a sample of older adults,

Anderson-Hanley et al. (2011) compared cycling through a

virtual course either alone or in the presence of on-screen rider avatars. In the latter condition, participants were

explicitly asked to outpace the avatars. The introduction of

the on-screen avatars increased cycling power output when compared to solo cycling condition. However, this effect

was observed only in participants who were classified as

high in competitiveness based on a self-report question- naire. A limitation of this study was that all participants

completed the solo cycling condition first and the com-

petitive situation second. Nevertheless, the findings suggest that competitiveness is an important moderating factor in

responses to VR.

Similar outcomes to Anderson-Hanley et al. (2011) were reached in a study by Snyder et al. (2012) who compared

two competitive situations while participants cycled in a

VR environment. In the virtual condition, the participants were informed that the avatar of the other rider was con-

trolled by the computer. In a live rider condition, the par-

ticipants were introduced to a confederate and were informed that the avatar speed was controlled by the

cycling speed of the confederate. Cycling performance,

measured as watts generated, was higher for the live rider condition than in the virtual rider condition. Again, this

difference emerged only in participants high in competi-

tiveness. No differences between the rider conditions emerged for participants low in competitiveness.

Competitive situations can be constructed within a vir-

tual environment in various ways. Nunes et al. (2014) devised three competitive modes based on whether par-

ticipants competed against themselves (i.e. a prior perfor- mance), against an individual chosen for them who is

superior, or against any individual chosen by the partici-

pant. Using a VR running task, all types of competitive situations enhanced physical exertion (as measured by

heart rate) and self-reported motivation when compared to

running on the virtual course alone. Evidence was also found that participants who were not initially competitive

still felt pressure to outperform the on-screen avatars.

However, similar to the conclusions reached by Anderson- Hanley et al. (2011) and Snyder et al. (2012), participants

who had a stronger preference for competitive situations

showed the highest task performance. A different approach to the use of another individual in

the virtual environment was reported by Oliveira et al.

(2015). Participants completed two conditions of a VR cycling task. In one condition, the participant self-selected

the intensity of the cycling trial. In the other condition,

participants were asked to follow a virtual cyclist. The virtual cyclist was set to a speed that matched the self-

selected intensity condition. No significant differences

were found between conditions on physiological effort or affective responses. Typically, an externally imposed

intensity results in an increase in negative affect. The

findings thus suggest that this affective ‘‘cost’’ is mitigated when participants match the imposed pace of a virtual

partner. However, further research is required to confirm

these findings. For example, order effects may have been a factor because all participants completed the self-selected

condition first and followed by the externally imposed

intensity condition.

5.2 User (Athlete) factors

The third component of the VR system is the athlete who is

engaging in the virtual sport. The characteristics of the

athlete user have the potential to mediate or moderate the effects of VR on performance and psychological outcomes.

Athlete user factors may include physical characteristics,

expertise and experience, and psychological characteris- tics. As shown in Table 2, the participants recruited in

research to date have been relatively homogenous. The

typical participant has been a young adult sampled from Western countries who are novice to the sport. It has been

Virtual Reality (2018) 22:183–198 189

123

suggested that using novices is advantageous because it results in a sample that is more physiologically equivalent

and their performance is less likely to be influenced by

prior learning (Hoffmann et al. 2014). However, it reduces the generalisation of findings to participants that are

younger or older or who compete at the elite level.

It is surprising that most studies have not reported comparisons between males and females given the docu-

mented gender differences in not only sport performance

but also in experience with computerised environments (e.g. computer games). Plante et al. (2003a) included

gender as a factor when examining the effects of VR on

mood during cycling. Females showed a larger difference in reported relaxation between the VR alone (no cycling)

and both the cycling alone and cycling with VR conditions

when compared to males. Plante et al. (2003b) also used a cycling task and reported gender differences in ratings of

energy. Males reported higher energy when cycling alone, cycling with VR, or experiencing VR alone than in a

baseline control condition that did not involve VR or

cycling. In contrast, females reported more energy in cycling alone or cycling with VR than when VR was used

without cycling or in the baseline condition. While pre-

liminary, there is some suggestion that females may be influenced more by the VR environment than males.

The preferences of the individual user may be an

important psychological factor that moderates outcomes. Legrand et al. (2011) assigned participants to either a

cycling task alone (no VR input), a self-selected VR task

(either jogging or cycling), or an externally imposed VR task (either jogging or cycling). All conditions improved

positive affect and reduced negative affect when assessed

by pre- and posttask subjective measures. The in-task subjective measures showed that participants in the self-

Table 2 Sample size and participant characteristics of studies investigating virtual reality in sport

Authors Years N Gender Age (in years) Experience type Location

Anderson-Hanley et al. 2011 14 Both M = 78.51 Novice U.S.A.

Range 60–99

Anderson-Hanley et al. 2012 79 Both M = 78.76 Novice U.S.A.

Anderson-Hanley et al. 2014 30 Both M = 79.5 Novice U.S.A.

Annesi and Mazas 1997 39 Both M = 37.7 Novice U.S.A.

Baños et al. 2016 109 Both M = 11.86 Novice Spain

Range 10–15

Chen et al. 2015 11 Both M = 24.5 Novice U.S.A.

Hoffman et al. 2014 15 Males M = 24.1 Novice France

Ijsselsteijn et al. 2004 24 Both M = 41.3 Novice Netherlands

Irwin et al. 2012 58 Females M = 20.54 Novice U.S.A.

Lee et al. 2012 275 Both 80.4% fell within 30–49 Experienced South Korea

Legrand et al. 2011 131 Both M = 19.31 Experienced France

Mestre et al. 2011 6 Not specified Range 19–25 Novice France

Murray et al. 2016 60 Female M = 20.20 Novice Australia

Range 18–30

Nunes et al. 2014 12 Both M = 33.91 Novice Brazil

Range 22–52

Oliviera et al. 2015 17 Male M = 31 Novice Brazil

Range 18–40

Plante et al. 2003a 88 Both M = 38.10 Novice U.S.A.

Range 20–67

Plante et al. 2003b 121 Both M = 18.58 Novice U.S.A.

Range 17–27

Sigrist et al. 2015 24 Both M = 26.1 Novice Switzerland

Range 21–33

Snyder et al. 2012 23 Females M = 19.2 Novice U.S.A.

Range 17–22

Vogt et al. 2015 22 Both M = 30.27 Novice Germany

190 Virtual Reality (2018) 22:183–198

123

selected VR task reported higher pleasure than the cycling

alone or the externally imposed VR task, which themselves did not differ. Autonomy or the appropriate matching of an

individual to a preferred sport may thus be important for

mood benefits when using VR. As noted above, individual preferences for task intensity may be another factor in that

using VR technology may reduce the negative impact of

performing at an externally imposed intensity (Oliveira et al. 2015).

5.3 Non-VR environment factors

The final component of the VR system, the real-world environment, has received no attention in research con-

ducted to date. Researchers have used a controlled indoor

environment and have kept key variables like temperature, humidity, and time of day constant or allowed them to vary

at random. Tenenbaum and Hutchinson (2007) noted that

the environment can be divided into physical and social components and a similar distinction can be made here. In

particular, based on research showing that the presence of

others in the virtual environment can influence perfor- mance and psychological states, it would be expected that

the presence of others in the real environment will also

have an influence. Further research is required to examine the effects of environmental factors and to determine the

relative strength of these factors when present virtually

versus when present in reality.

5.4 Concurrent and posttask outcomes

A summary of the key research aims and outcomes is

shown in Table 3. The majority of the outcomes have been

observed concurrently with the task, but some have been observed posttask (i.e. short-term and long-term effects;

see Fig. 1). Concurrent outcomes are those that influence

ongoing behaviour (e.g. performance, persistence, affective states, perceived exertion). For example, VR tasks that

induce competitiveness may induce short-term increases in

performance if the individual is running at a pace slower than a virtual competitor (Nunes et al. 2014). Posttask

outcomes will influence behaviour at a later time and are

thus independent of the ongoing interaction with the VR system (e.g. meeting performance goals, competition out-

comes). For instance, Annesi and Mazas (1997) showed

that an exercise program that used a VR cycling task increased adherence to the exercise program relative to

cycling alone.

Outcomes may also be divided into those related to task performance, physiological effects, and psychological

processes. As shown in Table 3, performance outcomes in

past research include adherence (Anderson-Hanley et al. 2014; Annesi and Mazas 1997; Irwin et al. 2012), distance

travelled or speed in the virtual environment (Hoffmann

et al. 2014; Ijsselsteijn et al. 2004; Murray et al. 2016; Nunes et al. 2014; Snyder et al. 2012), physical intensity

exerted (Anderson-Hanley et al. 2011; Chen et al. 2015;

Snyder et al. 2012), in-task persistence (Irwin et al. 2012), and strategy (Hoffmann et al. 2014). Physiological out-

comes have included heart rate (Nunes et al. 2014; Snyder

et al. 2012), oxygen consumption and blood lactate level (Oliveira et al. 2015), muscle fatigue (Chen et al. 2015),

and electroencephalogram (EEG) amplitude and frequency (Vogt et al. 2015). Psychological outcomes may relate to

behavioural intentions (Lee et al. 2012), cognitive func-

tions (Anderson-Hanley et al. 2012), motivation (Ijssel- steijn et al. 2004; Nunes et al. 2014), perceived pressure

(Ijsselsteijn et al. 2004), attentional focus (Baños et al.

2016; Mestre et al. 2011), and various positive and nega- tive feeling states.

The application of VR to sport has resulted in several

beneficial outcomes. When compared to control conditions, tasks that incorporate VR have shown improved adherence

(Annesi and Mazas 1997), better race strategy performance

(Hoffmann et al. 2014), higher cognitive functioning (Anderson-Hanley et al. 2012), improved mood and

reduced tiredness (Plante et al. 2003b), increased workload

(Chen et al. 2015), and higher enjoyment (Mestre et al. 2011; Murray et al. 2016). However, the control condition

used in most research has involved performance of the

sport on its own. This approach may be questioned because it does not control for the presence of an external stimulus

during the task. It is possible that the VR environment may

produce its effects because it distracts and diverts attention away from the task (Baños et al. 2016; Mestre et al. 2011),

rather than because it induces a sense of the presence or

includes elements of interactivity, which are the key fea- tures of a VR environment.

It is also noteworthy that better performance or psy-

chological outcomes have not always resulted when VR is used (e.g. Lee et al. 2012; Legrand et al. 2011) suggesting

that other factors may moderate its effectiveness. As noted

above and shown in Table 3, these factors may relate to the VR system or user, such as level of immersion (Ijsselsteijn

et al. 2004), competitiveness (Anderson-Hanley et al. 2011;

Nunes et al. 2014; Snyder et al. 2012), social presence (Irwin et al. 2012; Lee et al. 2012; Murray et al. 2016),

self-selection of tasks (Legrand et al. 2011), attentional

focus (Mestre et al. 2011), and the mood altering effects of the task itself (Plante et al. 2003b).

Performance and psychological outcomes may result

from the additive or interactive effects of the VR system. For example, a high level of immersion will enhance

motivation and performance (Ijsselsteijn et al. 2004).

However, immersion may be increased in different ways. It can be enhanced by using a more realistic VR environment

Virtual Reality (2018) 22:183–198 191

123

T ab

le 3

C h ar ac te ri st ic s o f th e d es ig n , ai m s, co n d it io n s, m ea su re s, an d k ey

fi n d in g s o f st u d ie s in v es ti g at in g v ir tu al

re al it y in

sp o rt

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

A n d er so n –

H an le y

et al .

2 0 1 1

Q u as i-

E x p er im

en ta l

T o ev al u at e th e ef fe ct

o f

so ci al

fa ci li ta ti o n an d

co m p et it iv en es s o n

cy cl in g in

o ld er

ad u lt s

(1 ) S ta ti o n ar y cy cl in g w it h V R

(2 ) S ta ti o n ar y cy cl in g w it h V R

an d o n -s cr ee n co m p et it o rs

C o m p et it iv en es s an d cy cl in g

ef fo rt

N o

T h e in tr o d u ct io n o f co m p et it o r

av at ar s in cr ea se d cy cl in g

in te n si ty

m o re

fo r co m p et it iv e

o ld er

ad u lt s th an

fo r th o se

w h o

w er e le ss

co m p et it iv e

A n d er so n –

H an le y

et al .

2 0 1 2

E x p er im

en ta l

T o d et er m in e if v ir tu al

cy cl in g w o u ld

re su lt in

g re at er

ex ec u ti v e

fu n ct io n , an d in cr ea se

b ra in -d er iv ed

n eu ro tr o p h ic

g ro w th

fa ct o r

(1 ) S ta ti o n ar y cy cl in g (2 )

S ta ti o n ar y cy cl in g w it h

in te ra ct iv e V R

to u rs

E x ec u ti v e fu n ct io n an d o th er

co g n it iv e fu n ct io n m ea su re s,

B M I, b o d y co m p o si ti o n ,

st re n g th , en er g y ex p en d it u re ,

an d b ra in -d er iv ed

n eu ro tr o p ic

fa ct o r

N o

V R cy cl in g to u rs sh o w ed

g re at er

ex ec u ti v e fu n ct io n in g an d

n eu ro p la st ic it y th an

cy cl in g o n

it s o w n ; V R

cy cl in g to u rs

h ad

a 2 3 %

re la ti v e ri sk

re d u ct io n in

m il d co g n it iv e im

p ai rm

en t

A n d er so n –

H an le y

et al .

2 0 1 4

E x p er im

en ta l

T o d et er m in e if h ig h er

ex ec u ti v e fu n ct io n w o u ld

p re d ic t cy cl in g

b eh av io u r o v er

a 3 -m

o n th

fo ll o w -u p

p er io d

(1 ) S ta ti o n ar y cy cl in g (2 )

S ta ti o n ar y cy cl in g w it h

in te ra ct iv e V R

to u rs

E x ec u ti v e fu n ct io n , se lf –

ef fi ca cy , p er ce iv ed

b en efi ts

an d b ar ri er s to

ex er ci se , so ci al

su p p o rt , m o ti v at io n , co g n it iv e

im p ai rm

en t, an d p h y si ca l

il ln es s

N o

E x er ci se

se lf -e ffi ca cy

an d

d ec li n in g ex ec u ti v e fu n ct io n at

p o st -i n te rv en ti o n w er e

as so ci at ed

w it h m o re

fr eq u en t

ex er ci se

d u ri n g fo ll o w -u p

A n n es i an d

M az as

1 9 9 7

E x p er im

en ta l

T o te st th e ef fe ct iv en es s o f

V R

o n in cr ea si n g

ad h er en ce , at te n d an ce ,

an d fe el in g st at es

(1 ) U p ri g h t b ic y cl e (2 )

R ec u m b en t b ic y cl e (3 )

R ec u m b en t b ic y cl e w it h V R

A tt en d an ce , ad h er en ce , ex er ci se –

in d u ce d fe el in g s, an d se lf –

m o ti v at io n

N o

V R

cy cl in g w as

ef fe ct iv e in

m ai n ta in in g ad h er en ce

to re g u la r cy cl in g ; ex er ci se –

in d u ce d fe el in g s w er e n o t

af fe ct ed

b y V R

cy cl in g

B añ o s

et al .

2 0 1 6

Q u as i-

E x p er im

en ta l

T o d et er m in e if a V R

w al k in g ta sk

cr ea te s

at te n ti o n al

d is tr ac ti o n

fr o m

b o d il y se n sa ti o n s in

o v er w ei g h t ch il d re n

(1 ) G ro u p (o v er w ei g h t, n o rm

al w ei g h t) (2 ) C o n d it io n (n o V R ,

V R )

H R , at te n ti o n al

fo cu s, ex er ci se –

in d u ce d fe el in g s, ra ti n g s o f

p er ce iv ed

ex er ti o n , en jo y m en t,

an d p re fe re n ce

N o

V R

d ec re as ed

fo cu s o n b o d il y

se n sa ti o n s an d in cr ea se d

ex te rn al

fo cu s fo r o v er w ei g h t

ch il d re n ; en jo y m en t ra ti n g s

w er e h ig h er

in th e V R

co n d it io n

C h en

et al .

2 0 1 5

E x p er im

en ta l

T o d et er m in e if V R

im p ro v es

w ei g h tl if ti n g

p er fo rm

an ce

an d ra ti n g s

o f p er ce iv ed

ex er ti o n

(1 ) V R (n o V R , 3 -D

st er eo , 2 -D

st er eo ) (2 ) W ei g h t o f li ft (l o w ,

m o d er at e,

h ig h ) (3 ) H ei g h t o f

li ft (l o w , m o d er at e,

h ig h )

M u sc le

fa ti g u e,

p o w er –

fr eq u en cy , ra ti n g s o f p er ce iv ed

ex er ti o n , an d p er ce iv ed

w o rk lo ad

N o

B ic ep

m u sc le

ac ti v it y an d

w o rk lo ad

w as

h ig h er

in b o th

V R

co n d it io n s th an

n o V R

co n d it io n ; ra ti n g s o f p er ce iv ed

ex er ti o n d id

n o t d if fe r ac ro ss

co n d it io n s

H o ff m an

et al .

2 0 1 4

E x p er im

en ta l

T o d et er m in e if V R

u si n g

an av at ar

to tr ai n a ra ce

st ra te g y w o u ld

im p ro v e

en er g y m an ag em

en t an d

ra ce

o u tc o m es

(1 ) V R

w it h n o av at ar

(2 ) V R

w it h av at ar

u si n g a fa st -s ta rt

ra ce

st ra te g y

V en ti la to ry

an d en er g y

ex p en d it u re

v ar ia b le s, ra ce

ti m e,

p o w er

o u tp u t, p ac e,

S te p M ax , an d ra ce

st ra te g y

N o

T ra in in g w it h an

av at ar

to u se

a fa st -s ta rt ra ce

st ra te g y

im p ro v ed

ra ce

st ra te g y p ro fi le s

an d ra ce

ti m e p er fo rm

an ce

at p o st -t es t an d re te n ti o n te st

192 Virtual Reality (2018) 22:183–198

123

T ab

le 3

co n ti n u ed

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

Ij ss el st ei jn

et al .

2 0 0 4

E x p er im

en ta l

T o d et er m in e if im

m er si v e

V R

en v ir o n m en ts

an d a

v ir tu al

co ac h in cr ea se

m o ti v at io n to

cy cl e

(1 ) Im

m er si o n (h ig h , lo w ) (2 )

V ir tu al

co ac h (w

it h , w it h o u t)

In tr in si c m o ti v at io n , H R , an d

av er ag e sp ee d

T h e IT C

S en se

o f P re se n ce

In v en to ry

W h en

V R

w as

m o re

im m er si v e,

m o ti v at io n an d av er ag e sp ee d

w er e in cr ea se d ; th e v ir tu al

co ac h re d u ce d p er ce iv ed

p re ss u re , te n si o n an d co n tr o l

Ir w in

et al .

2 0 1 2

E x p er im

en ta l

T o d et er m in e if m o ti v at io n

to p er si st w o u ld

b e

in fl u en ce d b y th e

p re se n ce

o f a p ar tn er

in a

co n ju n ct iv e o r co ac ti v e

si tu at io n

(1 ) In d iv id u al

(2 ) C o n ju n ct iv e

(3 ) C o ac ti v e

P er si st en ce , se lf -e ffi ca cy ,

in te n ti o n to

ex er ci se , ra ti n g s o f

p er ce iv ed

ex er ti o n , an d

in te n ti o n to

ex er ci se

N o

V R

co m b in ed

w it h a p ar tn er

sh o w ed

g re at er

ta sk

p er si st en ce

in co n ju n ct iv e

co n d it io n s th an

co ac ti v e

co n d it io n s w it h b o th

h ig h er

th an

n o p ar tn er

L ee

et al .

2 0 1 2

S u rv ey

T o in v es ti g at e th e

p sy ch o lo g ic al

ef fe ct s o f

p re se n ce

an d im

m er si o n

in V R

(1 ) V R

co n d it io n

P er ce iv ed

en jo y m en t, p er ce iv ed

v al u e,

an d b eh av io u ra l

in te n ti o n

A q u es ti o n n ai re

d es ig n ed

to m ea su re

te le p re se n ce

an d so ci al

p re se n ce

S o ci al

p re se n ce

ra th er

th an

th e

V R

te ch n o lo g y it se lf w as

re sp o n si b le

fo r en jo y m en t,

p er ce iv ed

v al u e,

an d

b eh av io u ra l in te n ti o n

L eg ra n d

et al .

2 0 1 1

E x p er im

en ta l

T o ex am

in e re g u la r

ex er ci se

v er su s a V R

cy cl in g an d V R

ru n n in g

ta sk

an d al so

th e ef fe ct o f

im p o se d v er su s se lf –

se le ct ed

V R

ta sk s o n

af fe ct

an d v al en ce

(1 ) B ic y cl e er g o m et er

w it h n o

V R (2 ) P ar ti ci p an t ch o ic e o f

V R cy cl in g o r ru n n in g (3 )

E x p er im

en te r al lo ca te d V R

cy cl in g o r ru n n in g

P o si ti v e af fe ct , n eg at iv e af fe ct ,

v al en ce

N o

M o o d b en efi ts fo ll o w in g th e ta sk

w er e o b se rv ed

re g ar d le ss

o f

co n d it io n ; a se lf -s el ec te d V R

ta sk

re su lt ed

in h ig h er

p o si ti v e

v al en ce

d u ri n g th e ta sk

th an

w h en

th e V R ty p e o f ta sk

w as

ex te rn al ly

im p o se d

M es tr e

et al .

2 0 1 1

E x p er im

en ta l

T o te st th e ro le

o f V R an d

a v ir tu al

co ac h o n

at te n ti o n al

fo cu s,

p er fo rm

an ce , an d

en jo y m en t

(1 ) N o V R

(2 ) V R

(3 ) V R

an d

fo ll o w in g v ir tu al

co ac h p ac er

P er ce iv ed

ex er ti o n , p h y si ca l

ac ti v it y en jo y m en t, at te n ti o n al

fo cu s, an d p er fo rm

an ce

(s p ee d ,

p o w er , p ed al li n g fr eq u en cy ,

an d H R )

N o

D is so ci at iv e at te n ti o n al

fo cu s

w as

g re at er

in th e V R

co n d it io n s an d en jo y m en t w as

g re at er

fo r V R co n d it io n th an

N o V R co n d it io n an d g re at er

fo r V R w it h v ir tu al

co ac h th an

V R

M u rr ay

et al .

2 0 1 6

E x p er im

en ta l

T o d et er m in e if th e

p re se n ce

o f o th er s in

an im

m er si v e V R

af fe ct s

p er fo rm

an ce , m o ti v at io n ,

an d af fe ct

d u ri n g an

ae ro b ic

ro w in g ta sk

(1 ) N o V R (2 ) in d iv id u al V R (3 )

co m p an io n V R

D is ta n ce , p o w er , st ro k es

p er

m in u te , H R , ex er ci se

th o u g h ts ,

p er ce iv ed

b en efi ts

an d b ar ri er s

to ex er ci se , ra ti n g s o f

p er ce iv ed

ex er ti o n , af fe ct ,

ar o u sa l, in tr in si c m o ti v at io n ,

an d en jo y m en t o f ex er ci se

N o

In d iv id u al

an d co m p an io n V R

re su lt ed

in b et te r ro w in g

p er fo rm

an ce

an d m o re

en jo y m en t w it h o u t an

in cr ea se

in p er ce iv ed

ex er ti o n ;

co m p an io n V R g ro u p

ex ce ed ed

in d iv id u al

V R g ro u p

in d is ta n ce

tr av el le d an d H R

Virtual Reality (2018) 22:183–198 193

123

T ab

le 3

co n ti n u ed

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

N u n es

et al .

2 0 1 4

E x p er im

en ta l

T o d et er m in e if th er e is

a d if fe re n ce

in p er fo rm

an ce

d u ri n g a

ru n n in g ta sk

in th e

p re se n ce

o f a v ir tu al

co m p et it o r

(1 ) S in g le p la y er

o n ly

(2 ) S in g le

p la y er

co m p et it iv e ag ai n st

o n es el f (3 ) C o m p et it iv e m o d e

ag ai n st a su p er io r ad v er sa ry

(4 ) C o m p et it iv e m o d e ag ai n st

an ad v er sa ry

ch o se n b y

p ar ti ci p an t

P er fo rm

an ce

(s p ee d , h ea rt b ea t,

an d d is ta n ce ), p er ce iv ed

ex er ti o n , p re fe rr ed

ex er ci se

co n d it io n

N o

P ar ti ci p an ts

re p o rt ed

th e

co m p et it iv e m o d e w as

m o re

m o ti v at in g ; p er ce iv ed

ex er ti o n

an d p er fo rm

an ce

w as

h ig h er

in co m p et it o r co n d it io n s th an

in si n g le

p la y er

m o d e

O li v ie ra

et al .

2 0 1 5

E x p er im

en ta l

T o d et er m in e if se lf –

se le ct ed

o r im

p o se d

ex er ci se

in te n si ty

an d

d u ra ti o n p ro d u ce

b et te r

cy cl in g p er fo rm

an ce ,

af fe ct iv e re sp o n se s, an d

en jo y m en t

(1 ) S el f- se le ct ed

in te n si ty

an d

d u ra ti o n w it h si n g le

v ir tu al

cy cl is t (2 ) Im

p o se d in te n si ty

an d d u ra ti o n w it h ad d it io n al

v ir tu al

cy cl is t

H R , o x y g en

co n su m p ti o n , b lo o d

la ct at e,

co n ce n tr at io n , ra ti n g s

o f p er ce iv ed

ex er ti o n , af fe ct ,

ar o u sa l, an d en jo y m en t o f

ex er ci se

N o

N o si g n ifi ca n t d if fe re n ce s in

p er fo rm

an ce

o r p sy ch o lo g ic al

o u tc o m es

ac ro ss

co n d it io n s

w er e o b se rv ed ; th er e w as

a tr en d to w ar d s h ig h er

en jo y m en t in

th e im

p o se d

se ss io n

P la n te

et al .

2 0 0 3 a

E x p er im

en ta l

T o in v es ti g at e w h et h er

V R

en h an ce s th e p o te n ti al

p o si ti v e ef fe ct s o f

cy cl in g

(1 ) S ta ti o n ar y cy cl in g at

m o d er at e in te n si ty

w it h n o V R

(2 ) P la y in g a V R

co m p u te r

b ic y cl e g am

e w it h n o ex er ci se

(3 ) S ta ti o n ar y cy cl in g at

m o d er at e in te n si ty

w it h V R

M o o d , p er ce iv ed

ex er ti o n , so ci al

d es ir ab il it y , en jo y m en t, an d

H R

N o

W h en

cy cl in g w as

p ai re d w it h

V R

m o o d in cr ea se d an d

ti re d n es s d ec re as ed

w h en

co m p ar ed

to cy cl in g al o n e

P la n te

et al .

2 0 0 3 b

E x p er im

en ta l

T o in v es ti g at e w h et h er

V R

en h an ce s th e

p sy ch o lo g ic al

b en efi ts

o f

cy cl in g al o n e

(1 ) W at ch

v id eo

si m u la ti n g a

cy cl in g ex p er ie n ce

(2 ) P la y in g

a V R

co m p u te r b ic y cl e g am

e w it h o u t ex er ci se

(3 ) S ta ti o n ar y

cy cl in g w it h o u t V R (4 )

S ta ti o n ar y cy cl in g w it h V R

M o o d , p er ce iv ed

ex er ti o n , so ci al

d es ir ab il it y , an d H R

N o

C y cl in g b u t n o t V R

g av e m o o d

im p ro v em

en ts

d ir ec tl y

fo ll o w in g th e ta sk ; b o th

V R

an d cy cl in g d ec re as ed

ti re d n es s;

V R

sh o w ed

p sy ch o lo g ic al

b en efi ts

h o u rs

af te r cy cl in g fo r fe m al es

o n ly

S ig ri st

et al .

2 0 1 5

E x p er im

en ta l

T o te st th e ef fe ct

o f

co n cu rr en t au g m en te d

fe ed b ac k o n le ar n in g an d

p er fo rm

an ce

o f a V R

ro w in g ta sk

(1 ) V is u al

fe ed b ac k (2 )

A u d io v is u al

fe ed b ac k (3 )

V is u o h ap ti c fe ed b ac k

S p at ia l er ro r, te m p o ra l er ro r,

co m fo rt , u se fu ln es s an d

ap p li ca b il it y o f fe ed b ac k , an d

st ra te g y fo r re ca ll in g ta u g h t

m o v em

en t

N o

P er fo rm

an ce

w as

b et te r fo r al l

g ro u p s in

fe ed b ac k co m p ar ed

to n o -f ee d b ac k tr ia ls ;

au d io v is u al

fe ed b ac k p ro d u ce d

b et te r le ar n in g an d g re at er

co m fo rt th an

v is u o h ap ti c

fe ed b ac k

S n y d er

et al .

2 0 1 2

E x p er im

en ta l

T o ex am

in e th e in fl u en ce

o f a v ir tu al

v er su s li v e

co m p et it o r o n cy cl in g

in te n si ty

an d en er g y

ex er ti o n

(1 ) C o m p et it iv en es s le v el

(2 )

L iv e o r v ir tu al

co m p et it o r

C o m p et it iv en es s an d cy cl in g

in te n si ty

(e n er g y o u tp u t, H R ,

an d sp ee d )

N o

P ar ti ci p an ts

w h o w er e h ig h ly

co m p et it iv e p ro d u ce d g re at er

cy cl in g in te n si ty

w h en

co m p et in g ag ai n st a li v e v er su s

a v ir tu al

co m p et it o r

194 Virtual Reality (2018) 22:183–198

123

as done by Ijsselsteijn et al. (2004). It can also be enhanced

if the individual has a high trait level to feel a greater sense of the presence. Interactions between external and indi-

vidual factors may also influence outcomes. For instance,

the introduction of a virtual competitor (VR environment factor) can increase performance (Nunes et al. 2014),

although the increase may only be observed if the individual

is competitive (athlete factor) as demonstrated in research (Anderson-Hanley et al. 2011; Snyder et al. 2012). Further

research is required to examine other interactive effects.

6 Future research directions and recommendations

The present review has highlighted issues that warrant further investigation. Most research to date has focussed on

VR tasks that involve aerobic sports (cycling, running, and

rowing). More research is required on the effectiveness of a VR environment for learning or improving the mechanics

of skill acquisition and performance in skill-based sports

(see Sigrist et al. 2015 for an example). The capacity for VR environments to be created in specific and reproducible

ways can allow for the training and assessment of skills and

decision-making processes. Some of the factors identified as important with aerobic sports (e.g. attentional focus,

competitiveness) may also be important in skill-based

sports when VR is used. Research is required to examine the generality of effects

with VR. Studies should include more diverse populations,

particularly experienced and elite athletes, children, and the elderly. In addition, research has also not examined rela-

tionships between performance in VR and real-world

environments. Identifying how the two situations differ and how they are the same could inform how VR influences

performance and psychological states. The transfer of

performance from the virtual environment to the real world has also not been tested, yet it seems an essential

requirement if VR is to be used as a training approach for

sport. Further research is required that aims to directly

manipulate psychological processes. For example, it has

been suggested that VR environments induce a dissociative attentional focus and that this may be related to affective

responses (Mestre et al. 2011). Baños et al. (2016) applied

this concept by asking overweight and normal weight children to walk on a treadmill while focussing their

attention on their physical feelings or while focusing their

attention on a virtual environment. Ratings of enjoyment were higher for the VR condition than the self-focused

condition, although there were no differences in perceived

exertion or feeling states. The findings are promising but are in need of replication and extension. Past research withT

ab le

3 co n ti n u ed

A u th o rs

Y ea rs

S tu d y d es ig n

A im

s C o n d it io n s

M ea su re s

Im m er si o n /

p re se n ce

m ea su re

K ey

fi n d in g s

V o g t et

al .

2 0 1 5

E x p er im

en ta l

T o te st th e ef fe ct

o f

ex er ci se

an d V R

im m er si o n o n co g n it iv e

p er fo rm

an ce

(1 ) S es si o n (a ct iv e cy cl in g ,

p as si v e au to m at ic

d ri v e)

(2 )

V R co n d it io n (f ro n t sc re en

o n ly , su rr o u n d w it h al l

sc re en s, co n tr o l w it h n o

sc re en s)

C o g n it iv e p er fo rm

an ce , H R ,

se n se

o f p re se n ce , an d E E G

am p li tu d e an d fr eq u en cy

Y es

S en se

o f p re se n ce

w as

as so ci at ed

w it h in cr ea se d E E G

ac ti v it y ; p re se n ce

w as

h ig h es t

in th e su rr o u n d V R

co n d it io n

d u ri n g ac ti v e cy cl in g ;

co g n it iv e p er fo rm

an ce

d id

n o t

d if fe r ac ro ss

co n d it io n s

V R v ir tu al

re al it y ; H R h ea rt ra te ; B M I B o d y M as s In d ex

Virtual Reality (2018) 22:183–198 195

123

non-VR tasks has also found that an external associative

focus enhances sport and exercise outcomes (e.g. Neumann and Heng 2011; Neumann and Piercy 2013). An external

associative focus involves focussing on the effects of

movements on the environment and the achievement of task goals (Neumann and Brown 2013; Stevinson and

Biddle 1999). Future research could thus use VR to induce

an external associative focus and examine its effectiveness in enhancing performance.

Further research is required to elucidate what factors are relevant to performance and affective outcomes.

Research using multiple measures or manipulations may

be particularly useful to determine the relative amounts of variance in performance attributed to different aspects of

the VR environment. In addition, different features of the

sport task should be varied. For example, intensity may be a particularly salient factor for aerobic sports. A higher

intensity level may switch attentional focus towards

internal physiological states (Stevinson and Biddle 1999) and result in individuals focusing attention away from the

VR environment. It may be possible to enhance atten-

tional focus on the virtual environment by requiring participants to follow a virtual partner as done by Oliveira

et al. (2015).

Finally, the nature of computer-based interactions is becoming more diverse and with a greater amount of

overlap between the different forms of technology and

their applications. The present review applied a definition of VR that required interactivity with the virtual envi-

ronment. However, it is acknowledged that researchers

are developing and testing systems that employ a virtual environment that the athlete responds to, even though the

behaviour of the athlete does not affect any feature of the

environment. For example, goalkeeping skills in penalty shots have been examined in both handball (Vignais et al.

2015), and soccer/football (Stinson and Bowman 2014).

In these applications, the goalkeeper viewed a virtual environment depicting an individual shooting a penalty

and was required to move their body in the predicted

direction of the ball. Their movements did not influence the action of the virtual penalty kick taker (e.g. moving

too early had no effect). Another instance that resembles

VR is the use of augmented reality. In such applications, a user has an indirect view of a physical, real-world

environment in which computer-generated input is added

to. The input may be visual, auditory, or other senses. This blending of real and virtual environmental elements

has yet to be extensively examined in sporting

applications. Based on the present review, recommendations can also

be made to ensure appropriate methodology and report in

studies. It is recommended that researchers:

1. Use the term virtual reality accurately and consistently

in reference to studies that have employed VR

technologies according to accepted definitions such as the one proposed here. The term should not be

confused with exergames, which refers to the more

general case of enhancing physical activity via inter- active computer game play. If interactivity with the

virtual environment is a particular feature that is to be

highlighted, such as in the present review, the term interactive VR may be used.

2. Report participants prior experience with VR in general

andwith the specificVR systembecause experience level may be an important factor that influences outcomes.

3. Use a measure of immersion or the presence as a

standard part of the protocol because these aspects are a core feature of VR, and the level of immersion has

emerged as an important factor that influences out-

comes (Ijsselsteijn et al. 2004; Vogt et al. 2015). Such measures include the Reality Judgement and Presence

Questionnaire (Baños et al. 2000) and the Presence

Questionnaire (Witmer and Singer 1998). 4. Provide full details of the VR system that is used.

These details include the name of the system or

software used, the participant point of view, the presence of others in the VR environment, the presence

of sounds in the VR environment, and the mechanisms

through which the participant interacts with the VR environment.

5. Report on relevant procedures that are important

psychologically, such as whether participants had choice over the type of VR task or discrete elements

within the task.

7 Conclusions

This review identified research studies that have investi-

gated the application of VR to sport. The research findings to date indicate that VR can be a promising adjunct to

existing real-world training and participation in sport.

A VR-based system for training and participation has several advantages such as enabling athletes to train

regardless of weather conditions, providing a means to

compete with others in a different geographic location, and allowing precise and replicable control over features of the

virtual environment. Future research would benefit from a

theoretical framework of VR application to sport (see Fig. 1). The present review has shown that the character-

istics of the individual user and system are important fac-

tors that can influence a range of performance, physiological, and psychological outcomes. By under-

standing the experience of when individuals are engaged in

196 Virtual Reality (2018) 22:183–198

123

sport within a VR environment, researchers, coaches, and

athletes will able to use the technology for the benefit of athletes and society in general.

References

Anderson-Hanley C, Snyder AL, Nimon JP, Arciero PJ (2011) Social facilitation in virtual reality-enhanced exercise: competitiveness moderates exercise effort of older adults. Clin Interv Aging 6:275–280. doi:10.2147/cia.s25337

Anderson-Hanley C, Arciero PJ, Brickman AM, Nimon JP, Okuma N, Westen SC, Molly EM, Pence BD, Woods JA, Kramer AF, Zimmerman EA (2012) Exergaming and older adult cognition: a cluster randomized clinical trial. Am J Prev Med 42:109–119. doi:10.1016/j.amepre.2011.10.016

Anderson-Hanley C, Arciero PJ, Barcelos N, Nimon J, Rocha T, Thurin M, Maloney M (2014) Executive function and self- regulated exergaming adherence among older adults. Front Hum Neurosci 8:989. doi:10.3389/fnhum.2014.00989

Annesi JJ, Mazas J (1997) Effects of virtual reality-enhanced exercise equipment on adherence and exercise-induced feeling states. Percept Motor Skill 85:835–844. doi:10.2466/pms.1997.85.3. 835

Australian Bureau of Statistics (2008) Defining sport and physical activity, a conceptual model. ABS Catalogue No. 4149.0.55.001

Baca A, Dabnichki P, Heller M, Kornfeind P (2009) Ubiquitous computing in sports: a review and analysis. J Sport Sci 27:1335–1346. doi:10.1080/02640410903277427

Baños RM, Botella C, Garcia-Palacios A, Villa H, Perpiñá C, Alcañiz M (2000) Presence and reality judgment in virtual environments: a unitary construct? Cyberpsychol Behav 3:327–335. doi:10. 1089/10949310050078760

Baños RM, Escobar P, Cebolla A, Guixeres J, Alvarez J, Francisco J, Botella C (2016) Using virtual reality to distract overweight children from bodily sensations during exercise. Cyberpsychol Behav Soc Netw 19:115–119. doi:10.1089/cyber.2015.0283

Briki W, Den Hartigh RJR, Markman KD, Micallef J, Gernigon C (2013) How psychological momentum changes in athletes during a sport competition. Psychol Sport Exerc 14:389–396. doi:10. 1016/j.psychsport.2012.11.009

Chen KB, Ponto K, Tredinnick RD, Radwin RG (2015) Virtual exertions: evoking the sense of exerting forces in virtual reality using gestures and muscle activity. Hum Factors 57:658–673. doi:10.1177/0018720814562231

Feltz DL, Kerr NL, Irwin BC (2011) Buddy up: the Köhler effect applied to health games. J Sport Exerc Psychol 33:506–526. doi:10.1123/jsep.33.4.506

Guy S, Ratzki-Leewing A, Gwadry-Sridhar F (2011) Moving beyond the stigma: systematic review of video games and their potential to combat obesity. Int J Hypertens. doi:10.4061/2011/179124

Hoffmann CP, Filippeschi A, Ruffaldi E, Bardy BG (2014) Energy management using virtual reality improves 2000-m rowing performance. J Sport Sci 32:501–509. doi:10.1080/02640414. 2013.835435

Ijsselsteijn W, de Kort Y, Westerink J, de Jager M, Bonants R (2004) Fun and sports: enhancing the home fitness experience. Lect Notes Comput Sci 3166:46–56. doi:10.1007/978-3-540-28643- 1_8

Irwin BC, Scorniaenchi J, Kerr NL, Eisenmann JC, Feltz DL (2012) Aerobic exercise is promoted when individual performance affects the group: a test of the Kohler motivation gain effect. Ann Behav Med 44:151–159. doi:10.1007/s12160-012-9367-4

Kim SY, Prestopnik N, Biocca FA (2014) Body in the interactive game: how interface embodiment affects physical activity and health behavior change. Comput Hum Behav 36:376–384. doi:10.1016/j.chb.2014.03.067

Larsen LH, Schou L, Lund HH, Langberg H (2013) The physical effect of exergames in healthy elderly–a systematic review. Games Health J 2:205–212. doi:10.1089/g4h.2013.0036

Laver KE, George S, Thomas S, Deutsch JE, Crotty M (2015) Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev 2. doi:10.1002/14651858.CD008349

Lee HG, Chung S, Lee WH (2012) Presence in virtual golf simulators: the effects of presence on perceived enjoyment, perceived value, and behavioral intention. New Media Soc 15:930–946. doi:10.1177/1461444812464033

Legrand FD, Joly PM, Bertucci WM, Soudain-Pineau MA, Marcel J (2011) Interactive-Virtual Reality (IVR) exercise: an examina- tion of in-task and pre-to-post exercise affective changes. J Appl Sport Psychol 23:65–75. doi:10.1080/10413200.2010.523754

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta- analyses of studies that evaluate healthcare interventions: explanation and elaboration. Br Med J 339:b2700. doi:10.1136/ bmj.b2700

Meline T (2006) Selecting studies for systematic review: inclusion and exclusion criteria. Contemp Issues Commun Sci Disord 33:21–27

Mestre DR, Ewald M, Maiano C (2011) Virtual reality and exercise: behavioral and psychological effects of visual feedback. Stud Health Technol Inf 167:122–127. doi:10.3233/978-1-60750-766-6-122

Meyerbröker K, Emmelkamp PMG (2010) Virtual reality exposure therapy in anxiety disorders: a systematic review of process-and- outcome studies. Depress Anxiety 27:933–944. doi:10.1002/da. 20734

Mueller FF, Stevens G, Thorogood A, O’Brien S, Wulf V (2007) Sports over a distance. Pers Ubiquit Comput 11:633–645. doi:10. 1007/s00779-006-0133-0

Murray EG, Neumann DL, Moffitt RL, Thomas PR (2016) The effects of the presence of others during a rowing exercise in a virtual reality environment. Psychol Sport Exerc 22:328–336. doi:10.1016/j.psychsport.2015.09.007

Neumann DL, Brown J (2013) The effect of attentional focus strategy on physiological and motor performance during a sit-up exercise. J Psychophysiol 27:7–15. doi:10.1027/0269-8803/a000081

Neumann DL, Heng S (2011) The effects of associative and dissociative attentional focus strategies on muscle activity and heart rate during a weight training exercise. J Psychophysiol 25:1–8. doi:10.1027/0269-8803/a000011

Neumann DL, Piercy A (2013) The effect of different attentional strategies on physiological and psychological states during running. Aust Psychol 48:329–337. doi:10.1111/ap.12015

Nunes M, Nedel L, Roesler V (2014) Motivating people to perform better in exergames: Competition in virtual environments. In: Proceedings of the 29th annual ACM symposium on applied computing, ACM, New York. pp 970–975. doi:10.1145/ 2554850.2555009

Oliveira BRR, Deslandes AC, Nakamura FY, Viana BF, Santos TM (2015) Self-selected or imposed exercise? A different approach for affective comparisons. J Sports Sci 33:777–785. doi:10.1080/ 02640414.2014.968191

Peng W, Crouse JC, Lin JH (2013) Using active video games for physical activity promotion: a systematic review of the current state of research. Health Educ Behav 40:171–192. doi:10.1177/ 1090198112444956

Plante TG, Aldridge A, Bogden R, Hanelin C (2003a) Might virtual reality promote the mood benefits of exercise? Comput Hum Behav 19:495–509. doi:10.1016/S0747-5632(02)00074-2

Virtual Reality (2018) 22:183–198 197

123

Plante TG, Frazier S, Tittle A, Babula M, Ferlic E, Riggs E (2003b) Does virtual reality enhance the psychological benefits of exercise? J Hum Movement Stud 45:485–507. doi:10.1037/ e314842004-001

Plante TG, Cage C, Clements S, Stover A (2006) Psychological benefits of exercise paired with virtual reality: outdoor exercise energizes whereas indoor virtual exercise relaxes. Int J Stress Manag 13:108–117. doi:10.1037/1072-5245.13.1.108

Ranganathan R, Carlton LG (2007) Perception-action coupling and anticipatory performance in baseball batting. J Motor Behav 39:369–380. doi:10.3200/JMBR.39.5.369-380

Reynolds JE, Thornton AL, Lay BS, Braham R, Rosenberg M (2014) Does movement proficiency impact on exergaming perfor- mance? Hum Movement Sci 34:1–11. doi:10.1016/j.humov. 2014.02.007

Sherman WR, Craig AB (2002) Understanding virtual reality: interface, application, and design. Elsevier, San Francisco

Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2015) Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp Brain Res 233:909–925. doi:10.1007/s00221-014-4167-7

Slater M (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos T R Soc B 364:3549–3557. doi:10.1098/rstb.2009.0138

Snyder AL, Anderson-Hanley C, Arciero PJ (2012) Virtual and live social facilitation while exergaming: competitiveness moderates exercise intensity. J Sport Exerc Psychol 34:252–259

Steuer J (1992) Defining virtual reality: dimensions determining telepresence. J Commun 42:73–93. doi:10.1111/j.1460-2466. 1992.tb00812.x

Stevinson CD, Biddle SJH (1999) Cognitive strategies in running: a response to Masters and Ogles (1998). Sport Psychol 13:235–236. doi:10.1123/tsp.13.2.235

Stinson C, Bowman DA (2014) Feasibility of training athletes for high-pressure situations using virtual reality. IEEE Trans Vis Comput Gr 20:606–615

Tenenbaum G, Hutchinson JC (2007) A social-cognitive perspective of perceived and sustained effort. In: Tenenbaum G, Eklund RC (eds) Handbook of sport psychology, 3rd edn. Wiley, New Jersey, pp 560–573

Tirp J, Steingrover C, Wattie N, Baker J, Schorer J (2015) Virtual realities as optimal learning environments in sport—a transfer study of virtual and real dart throwing. Psychol Test Assess Model 57:57–69

Vignais N, Kulpa R, Brault S, Presse D, Bideau B (2015) Which technology to investigate visual perception in sport: video vs. virtual reality. Hum Mov Sci 39:12–26. doi:10.1016/j.humov. 2014.10.006

Vogt T, Herpers R, Scherfgen D, Strüder HK, Schneider S (2015) Neuroelective adaptations to cognitive processing in virtual environments: an exercise-related approach. Exp Brain Res 233:1321–1329. doi:10.1007/s00221-015-4208-x

Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a Presence Questionnaire. Presence-Teleop Virt 7:225–240. doi:10.1162/105474698565686

198 Virtual Reality (2018) 22:183–198

123

Journal of Athletic Training 2020;55(9):902–910 doi: 10.4085/1062-6050-0540.19 ! by the National Athletic Trainers’ Association, Inc www.natajournals.org

Current Concepts

‘‘To Tech or Not to Tech?’’ A Critical Decision-Making Framework for Implementing Technology in Sport Johann Windt, PhD, CSCS*§; Kerry MacDonald, PhD†; David Taylor, MSc‡; Bruno D. Zumbo, PhD§; Ben C. Sporer, PhD*§; David T. Martin, PhD||

*Vancouver Whitecaps FC, BC, Canada; †Volleyball Canada, Ottawa, ON; ‡United States Olympic and Paralympic Committee, Colorado Springs, CO; §University of British Columbia, Vancouver, Canada; ||Australian Catholic University, Belbourne

The current technological age has created exponential growth in the availability of technology and data in every industry, including sport. It is tempting to get caught up in the excitement of purchasing and implementing technology, but technology has a potential dark side that warrants consideration. Before investing in technology, it is imperative to consider the potential roadblocks, including its limitations and the contextual challeng- es that compromise implementation in a specific environment. A thoughtful approach is therefore necessary when deciding

whether to implement any given technology into practice. In

this article, we review the vision and pitfalls behind technology’s

potential in sport science and medicine applications and then

present a critical decision-making framework of 4 simple

questions to help practitioners decide whether to purchase

and implement a given technology.

Key Words: analytics, measurement, wearable devices, global positioning systems

T echnology is here to stay—not just in sport but in virtually every discipline. This special issue focuses on training load, recovery monitoring, and manage-

ment, and in nearly every article, readers will find examples of how technology can be used in these areas. External loads can be monitored through global positioning systems (GPS), inertial measurement units (IMUs), optical tracking systems, and so on.1 Internal loads may be captured with heart-rate monitors, lactate measurements, and more.2

Recovery states may be measured with devices ranging from low-tech wellness surveys3–5 to more high-tech solutions, such as heart-rate variability6,7 or force-plate testing.8,9 Currently, we are seeing new technological solutions with potential sporting applications, such as implantable devices,10 markerless motion capture,11,12

breath analysis,13 smart garments, biomechanical insoles, and skin sensors.14 In this technological age, sports science practitioners must critically appraise the plethora of options available and make informed decisions about evaluating and adopting technology in their specific contexts. These context-specific questions demand a critical evaluation of the case for the intended use and the available evidence that supports (or does not support) technological implementa- tion. Our aim in this article is to provide a simple, foundational framework to aid practitioners in that critical decision-making process.

This article is divided into 3 parts: (1) a vision for what technology can provide and why we should be excited about its potential, (2) a warning about the potential dark side of technology and the pitfalls that can derail its successful implementation, and (3) a critical decision-

making framework consisting of 4 key questions to ask before purchasing a new technology.

PART 1: THE BRIGHT SIDE—A VISION FOR TECHNOLOGY

Excitement is the most appropriate response when taking an optimist’s view of technology in sport. In this section, we outline just a handful of benefits practitioners can expect from successful technological implementation.

Benefit 1: Improving and Off-Loading Data Collection—An Example From Pro Football

Technology can improve measurement precision and automate the process so that practitioners do not have to manually record data as they did in the past. For example, understanding the match demands to which athletes are exposed is foundational in the sport sciences.15–17 Football is no exception, and time-motion analyses to understand football players’ physical outputs (eg, total distance run in a match, time spent at different speeds) and physiological responses have been performed for decades.18–20 Before the technological advances that are commonplace today, these time-motion analyses were performed using tape-recorded commentaries, video recordings, and film analyses. All of these notational analysis processes were extremely time consuming, often limiting researchers’ ability to examine more than a small number of players in a defined number of matches.21 Technological advances, both through wearable devices22 and optical tracking systems,21 now provide these physical output measures to researchers and practitioners in near real time.23,24 Although these systems are not without

902 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

error and vary among technological providers and sys- tems,25 many provide more accurate physical output data than estimates derived from manual notational analyses based on video and can supply these data on all players simultaneously and in near real time. In this way, technology and its efficiency have off-loaded weeks and months of practitioner and researcher time. If used wisely, this regained time allows a deeper dive into the information and may inform practice more thoroughly. For example, advances in optical tracking and wearable technology have now allowed a better understanding of how physical exertion in football players relates to such contextual variables as player position, stage of play, and teams or players being in or out of possession.26–28

Benefit 2: Sport-Specific Load Measures—A Volleyball Journey

The prevalence of chronic injuries in volleyball players is known to be high.29,30 Of these injuries, jumper’s knee (ie, patellar tendinopathy) is the most common. The injury mechanism is fundamentally an overload of the extensor mechanisms of the knee joint.31 With the introduction of IMUs that measure athlete jump counts, the sport-specific load requirements of the knee extensors could be quantified for the first time without manual video annotation of jump counts for all players.32

One author (K.M.) used his dual roles as a researcher and volleyball coach to scientifically evaluate such a technology and implement simple heuristics to inform decision making in coaching. As a researcher, he performed validation work on a wearable IMU for measuring jump count32 to ensure that he could rely on the jump-count data being provided by the IMU software. Specifically, the jump counts from the accelerometers related very closely to the jump counts extracted from manual video notation, so he was comfort- able that the errors were few enough to reasonably inform practice. As a coach, measuring jump loads for all players in training and matches facilitated a better understanding of the position-specific training and match demands and individualized athlete load management throughout the season. After a retrospective assessment of training and match demands, he prospectively planned and prescribed individual jump loads. This prospective prescription may have improved the capacity of the players’ tendons to withstand the sports demands and prevent the development or flare-ups of such chronic injuries as jumper’s knee. In his case, this informed decision-making process helped to mitigate the prevalence of overuse injuries (zero practice sessions or games missed due to overuse injuries) in the team’s volleyball players and culminated in a national championship.

Although not all stories end in a championship, technological implementation allows for sport-specific and movement-specific load quantification that can inform practitioners’ workload, recovery, and return-to-sport decisions. A multitude of factors affect the onset of any injury, yet an informed approach to load is certainly a beneficial addition to any injury-mitigation strategy.

Benefit 3: A 3608 View of the Athlete

Performance is multifactorial and requires adequate physical, mental, technical, and tactical expertise to

compete at the elite level. The contribution of each element depends on the sport’s demands and the characteristics of the individual athlete. Furthermore, sport is dynamic and ever changing as athletes pursue multiple phases over the course of a single season, including training, competition, and recovery. Sleep, recovery, nutrition, social factors, and lifestyle can all affect athletes’ responses to and outcomes in training and performance.33,34

Technology allows for the rapid collection and analysis of data from many of these areas. The ability to integrate data streams enables practitioners to better understand how one factor affects another by providing a holistic perspec- tive of the athlete. It also permits information to be shared across disciplines, blending injuries with training load, medicine with physiology, and physical with technical and tactical performance. Where limitations once existed in storing and processing vast amounts of information safely and in a time-efficient manner, technological advancements have reduced many of the challenges involving costs, computing speed, and intelligence tools. Although pitfalls still exist (see the next section), technology allows practitioners and teams to provide holistic perspectives on athlete performance when using a strategic approach.

PART 2: THE POTENTIAL DARK SIDE—A WARNING TO THE WISE

The data life cycle may be summarized broadly as plan ! collect ! analyze ! communicate. Each subsequent step relates closely to the intended use of the information, as determined by a thoughtful plan underpinning the technology’s implementation. A problem at any stage of this life cycle can be fatal for the successful use of any technology. Failed technological implementations can have lasting ramifications, so considering the following potential pitfalls is important.

Pitfall 1: Not All Promises of Technology Are Kept

Underlying all stages of the data life cycle is a belief that the data are trustworthy enough to collect and interpret. However, some of the bold promises made by technology companies may not actually be true. In these instances, failed promises may result in poor data quality (eg, measurement error is too large) that challenges a practi- tioner’s ability to interpret any signal amid the noise. The failures could also stem from black-box algorithms that summarize the data and produce unactionable, uninterpret- able outputs.

Scientists have explored technological devices in an attempt to better understand the validity and reliability of the data from these emergent technologies. As such, different technologies are known to have inherent limita- tions: for example, the ability of GPS technology to accurately measure high-speed running velocities,22,35,36 the sensitivity of heart-rate variability measurement,37 and the subsequent requirement of rigid, standardized testing procedures or the effectiveness of wrist-based sleep monitoring compared with the criterion standard of polysomnography testing.38 Although these examples include some technological devices that have published validity-related evidence, it is important to note 2 items. First, none of these devices and the data they provide are perfect; all come with inherent measurement error. Second,

Journal of Athletic Training 903

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

most consumer devices have little scientific evidence for their accuracy, validity, and reliability,14 so a prudent practitioner should approach any new technological device with a healthy dose of skepticism.

Pitfall 2: Technology Transforms Into a Dust Collector If It Cannot Be Implemented

The best technologies are useless if they are not implemented in a way that informs decision making or changes practice. Given that technological implementation may require sizable investments of financial and human resources, understanding the burdens on time and staff resources that implementation will require is crucial. If the burden on staff is too great, practitioners may be stretched beyond their skill sets, be forced into uncomfortable situations, or have unrealistic time constraints placed on them, negatively affecting the feasibility and quality of data collection. If the staff are not educated about the potential benefits, lack the desire to collect the data appropriately, and do not believe the technology will provide useful information, the investment is flawed before it begins.

Pitfall 3: Technology Does Not Necessarily Provide the Right Data or the Raw Data

A high-tech solution and precise data collection do not inherently mean that the right data are being collected. What constitutes the right data in a particular setting depends on several contextual and organizational factors, such as the questions being asked by different practitioners within the organization and how the organization’s decision-making structure allows data to inform decisions.

Even if the right data are being collected, it is vital to understand what type of data the technology will provide. Many technologies come with a software package that delivers a dashboard or printable report of the data collected. It is important to consider whether these standard reports or dashboards analyze the data in a way that reflects the user’s needs and corresponds to the original plan. The analyze portion of the data life cycle implies that the technology analyzes the data in the way you need or that you can access and analyze the data yourself in accordance with those needs if the technology does not provide the answers you are seeking. In these instances, it is important to understand whether a technology provides access to the raw data so that you can perform the appropriate analyses in accordance with your original plan. If the technology does not provide such access and only reports summary findings based on proprietary algorithms, the ability of researchers and practitioners to analyze data in the ways they need may be compromised.

Pitfall 4: Technology Does Not Inherently Communicate a Message

Even when technology is introduced and data are collected consistently in an applied environment, the data have just made it through part of the data life cycle—they must still be analyzed and disseminated in accordance with the plan. Technology itself does not inherently communi- cate to decision makers. Although some technological devices are accompanied by software tools that provide reports or dashboards that summarize the underlying data,

the message delivered to decision makers must be readily interpretable by the end user, which can include high- performance team members, coaching staff, or manage- ment, and answer the specific questions that were planned. What appropriate communication and dissemination look like, therefore, depend on the intended use of the technology and the context in which it is implemented. Understanding end users’ requirements, interests, and necessary decisions is crucial so that information can be tailored into a clear, concise message. Crucially, these steps must be taken in each environment and are not accom- plished simply by having purchased a given technology.

PART 3: A CRITICAL DECISION-MAKING FRAMEWORK

Asking the right questions before jumping into a new technological investment can help guide practitioners and researchers to the vision of the technology while avoiding some of the common pitfalls. Unsurprisingly, several frameworks for integrating technology into sport have been proposed in the literature,39,40 and we strongly encourage readers to explore and critically think through other frameworks in addition to those presented here. In our critical decision-making framework, we pose 4 questions, all of which should be answered affirmatively before arriving at a decision to purchase a given technology (Figure 1). Each of these 4 questions, important follow-up questions, sources of evidence for finding appropriate answers, and key take-home messages are discussed in the following sections, detailed in Figure 1, and summarized in the Table.

Question 1: Would the Promised Information Be Helpful?

New technologies arrive on the market daily, many of which may pique the interest of curious and intelligent individuals. These technologies often come with bold claims, savvy marketing, and grand promises. In the sport sciences, these claims may include accurate injury prediction or ‘‘1-stop shops’’ for understanding an athlete’s fatigue and recovery status. We propose that the first question practitioners should ask when encountering a technology and engaging with such claims is would the promised information be helpful? The ability to extract new information can be exciting, yet this does not mean the information will help to inform decision making for practitioners in their specific contexts.

To answer this question, individuals should consider what specific question will be answered or which decision will be informed. This is an extremely important point, as technology must inform practice to be useful. When a new technological opportunity is considered, at least 1 key decision should be informed by the available information or 1 question should be answered. This is the premise of the first (and most important) stage of the data life cycle: planning. To plan effectively, practitioners must understand their specific contexts so they address relevant and pertinent questions that end users need to answer. Ideally, a need may already have been identified within the practice for more information to be collected, in which case investing in a technology that provides that specific data makes more sense. Considering 2 contexts from earlier in this article,

904 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

wearable technology may provide insight for team-sport practitioners, but this looks different in soccer, where GPS data may be most valuable to understand athlete distances and speeds, than volleyball, where an accelerometer that provides information about jump counts and distances may be deemed less important (and not measurable with GPS indoors). Note that in each of these instances, a need is a question that must be answered or a decision that must be informed, rather than a specific technology that the organization believes is in itself necessary.

Take-Home Suggestions. ! Start with the end in mind. Understand the decision you

want to inform and which data from the technology you

will need to extract and communicate in order to inform that decision.

! Explore existing paths. Is there an existing data stream you can use instead or a better alternative?

Question 2: Can You Trust the Information You Will Be Getting?

When researchers speak about technology, the discussion typically includes aspects of measurement error, reliability, responsiveness, and validity. Beyond all these technical terms, practitioners need to know whether they can take

Table. Unpacking Each Question Within the Critical Framework Through Follow-Up Questions, Sources of Evidence, and Take-Home Messages

Would the promised information be helpful?

Can you trust the information you

will be getting?

Can you integrate, manage, and analyze the data effectively?

Can you implement the technology in

your practice?

Follow-up questions

What question will you answer or what decision will you inform?

Has a need already been identified for the promised information?

How much validity-related evidence is available regarding the new technology?

Are you confident enough with the limitations of the technology to inform practice?

In what format and by what means is information from the technology delivered, and how much cleaning needs to be done to integrate it with other measurements?

Do you have the analytical resources to handle and analyze the data?

What burden is placed on athletes and practitioners to collect the data?

Does your culture allow for technology to be implemented and data to be collected, and will the technology affect the culture?

Does your context allow for data to inform and alter practice?

Sources of evidence

Understand the challenges of your own context.

Consult with other researchers and practitioners who have faced similar questions and challenges.

Scientific literature surrounding the validity- related evidence for the technology

Internal validation and reliability

Professional network

Data samples from the company, short-term trials

Internal discussions or methodologic and statistical consultancy

Professional network

Qualitative scientific evidence Internal communication (formal

and informal) and education

Take-home messages

Start with the end in mind. Evaluate the existing

environment and infrastructure to see whether you need new technology to get the information.

Evaluate continually. Consider the consequences. Pilot where possible. Partner where appropriate.

Plan ahead. Educate practitioners involved in

collection on proper formatting and process.

Automate processes where possible.

Audit data and proactively set up quality controls.

Understand the implementation context.

Look for invisible monitoring opportunities.

Build technological implementation into existing routines.

Figure 1. A critical decision-making framework for integrating technology in sport.

Journal of Athletic Training 905

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

information from a given technology and be confident in making a decision based on the evidence it provides. We believe the principles of unified validity theory can help guide researchers and practitioners in trying to answer this question of trustworthiness.

A Brief Overview of Unified Validity Theory. Ground- ed in the work of Loeveringer41 and Cronbach and Meehl,42

Samuel Messick posited and promoted a unified view of validity theory.43–45 In place of validity types, the following definition of unified validity theory was proposed by Messick and adopted in the Standard for Educational and Psychological Testing: ‘‘Validity is an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropri- ateness of interpretations and actions based on test scores or other modes of assessment.’’46

Unpacking this definition reveals 3 primary ways in which unified validity theory differs from other common and more traditional views of validity:

1. Validity is about claims and inferences that can be made, not about measures.

2. Validity evidence has multiple sources, and the aim is an integrated evaluative judgment (Figure 2).

3. Validity invites consideration of the consequences associated with technology and the data it provides.

How Does Validity Theory Translate to Adopting a New Technology? Unified validity theory provides prac- titioners and scientists with a lens to look through as they consider different measures. When we look through this lens, we see that thinking about measurements from technology is similar to thinking about science. The answers to most of our questions are more nuanced than yes versus no or valid versus invalid. Instead, we use terms such as it depends, to a degree, or in this specific context. This sets the stage for a practitioner considering a new technology to evaluate the technology on a continuum in terms of his or her specific context.

As an integrative, evaluative judgement, practitioners should examine all the available sources of evidence regarding a given technology and the specific metric it provides. Some answers may be found in the peer-reviewed literature, or the practitioner may have to pilot data internally. Reaching out to colleagues who have already

implemented these technologies may offer opportunities to discuss their internal validity-related evidence.

The consequences of testing are a final consideration that unified validity theory emphasizes and that practitioners should carefully consider. Implicit and explicit conse- quences are inherent when measuring and testing some- thing by implementing a technology. Athletes and practitioners will consider the quality being measured as important, athletes may train to improve that given quality, and decisions may be based more heavily on the provided data than on other pieces of information. These intended and unintended consequences can be positive or negative but should be considered carefully.

This process and evaluation must be performed on each of the different metrics that a practitioner hopes to use to inform decision making. Returning to our volleyball example, the IMU provides a more accurate measure of jump count than does video notation. However, this same IMU also provides measures of jump height and ground reaction forces. Although these measures may theoretically be linked to overuse injuries and performance, they should each be further investigated. In this case, video notation would not be the appropriate comparison measure, and more advanced biomechanical analyses and equipment would be preferable.

Ultimately, no technology, or the data it provides, is perfectly trustworthy. The practitioner faces this question: Given all the information at his or her disposal, are the limitations of the technology minimal enough that it can still inform decision making?

Take-Home Suggestions. ! Evaluate continually. View trust in one’s data as an

ongoing endeavor to judge how trustworthy the technol- ogy and data are on a spectrum using all the available sources of validity-related evidence.

! Consider the consequences of testing. What potential consequences, intended and unintended, could introduc- ing the technology have for athletes and practitioners?

! Pilot test where possible. If practitioners can gain early access to the technology before purchasing, they can conduct preliminary analyses of the data before purchas- ing.

! Partner where appropriate. When in-house expertise is not sufficient for examining certain aspects of the data,

Figure 2. Some sources of validity-related evidence.

906 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

collaborate with a research laboratory, university, or third-party company to facilitate analyses of the trustworthiness of the technology.

Question 3: Can You Integrate, Manage, and Analyze the Data Effectively?

A 3608 view of an athlete’s training, recovery, lifestyle, and so on is a major potential benefit of technological implementation. If a technology and the data it provides are deemed trustworthy enough to be added to this holistic athlete view, the next step is to understand how the data will be extracted, integrated with other data sources, and analyzed in a meaningful way.47 Different technologies provide different levels of granularity to the data, and the means of data extraction can vary from manual download- ing of files (eg, spreadsheets) to application programming interfaces that allow automated data extraction. The extracted data may also need additional cleaning before analysis and modification to be integrated with other data sources. It is vital to understand how much time will be needed to extract and clean the data; although these processes (eg, manual downloads and spreadsheet data management) may be common, they limit scalability and may preclude successful and sustainable implementation of a given technology.

A well-known truism in the data-science realm is that data scientists spend most of their time cleaning and preparing data for analysis. Given the complexity and challenges inherent in combining data from disparate technologies, practitioners must consider whether they have the expertise on their team or at their disposal to create a system that brings their data together. This may be accomplished through in-house data-science personnel, third-party athlete-management systems, or external con- sulting agencies. Without this expertise, it is very challenging to combine data sources to create a holistic athlete profile.

Once the data are collected and combined, analysis presents its own challenges. Over the last few decades, intensive longitudinal data have become increasingly common in elite sport settings. These data present a specific set of challenges and assumptions inherent to repeated-measures data, and in many instances, more sophisticated analyses are recommended to deal with these challenges.48,49 At least in the workload injury field, the authors of a methodologic review50 identified that statistical approaches to adequately address these challenges when investigating the question of how workload data relate to injury risk have not been applied to many intensive longitudinal data sets. Statistical in-house or outsourced expertise can help ensure that the statistical approaches applied are appropriate for the data complexity.

To understand the demands of accessing, cleaning, integrating, and analyzing the data that a new technology will provide, it is highly prudent to ask the company for a free trial and access to their data streams. Discussions with other practitioners in the industry who already use the technology may also be fruitful to paint a realistic picture of the data-management demands.

Take-Home Suggestions. ! Plan ahead: Obtaining data samples from prospective

companies ahead of time helps to ensure that the

processes and systems can be tested and evaluated before a technology is introduced.

! Educate: By training practitioners in basic principles of data collection,51 many of the data-cleaning challenges can be proactively prevented.

! Automate: Software solutions (eg, Alteryx [Irvine, CA], Fivetran [Oakland, CA], Matillion [Manchester, UK]) and open-source coding platforms (eg, R [Vienna, Austria], Python [Wilmington, DE]) can enable data scientists to streamline and automate many processes, thereby reducing the amount of time required to manually input, download, and edit data.

! Audit: Set up data-audit or quality-control checks to ensure the data are clean and appropriately combined and then respond appropriately when you find mistakes and outliers.

Question 4: Can You Implement the Technology in Your Practice?

The fourth and final question to consider is whether a technology can realistically be implemented in your specific sporting context. In the research arena, we know that injury-prevention protocols that are effective in a controlled trial setting may fail to deliver the same results in a real-world environment because the effects are largely dependent on the adoption and implementation of the program.43–45 In the same way, even a near-perfect technology may fail in an environment where it encounters an implementation problem.

Implementation failures may occur anywhere along the data-science pipeline. The challenges in data analysis were largely addressed in question 3, but implementation challenges are especially pivotal to consider at the data- collection and data-dissemination steps.

Implementation challenges in data collection stem from increased practitioner or athlete burden. The demand on staff and athletes alike to collect data can be significant. It is not uncommon for the rollout of a new technology to be one more item in a long list of responsibilities for staff. Athletes are also not always burden free when it comes to the implementation process. It is important to consider what the athletes will be asked to do, the collective burden of technology and data collection on athletes as a whole, and how the athletes will perceive the new technologies. One must consider the possible ramifications of this increased staff and athlete burden and whether the technology’s potential benefits outweigh the cost of implementation.

The flexibility, mentality, and willingness of the people in an environment to adopt new practices can determine whether implementation challenges in data-informed deci- sion making arise. Certain sports have already embraced technology, whereas others may be deemed resistant to technological innovations. It is therefore essential to consider if members of your sporting culture will accept a specific technology in their environment. For numerous reasons, an organization may not want the data or may not be keen to have the answers that the data could provide. Also, deep-rooted doubt in the reliability or validity of the data may prevail, despite the best available evidence. Many sporting cultures resist changing the way things have always been done, so technological implementation may be seen as changing or modifying their sport.

Journal of Athletic Training 907

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

The successful implementation of any technology requires careful consideration of the time and resources required from practitioners and athletes, the process and procedures that need to be in place to minimize the burden from the technology, and the communication and decision- making channels whereby the analyzed data will be delivered and used to inform practice.

Take-Home Suggestions. ! Understand and improve the implementation context.

Consider the burden and challenges on practitioners and athletes at all points across the data pipeline. Educating practitioners on the rationale and benefits associated with technology and empowering them in their roles may facilitate ‘‘buy-in’’ and more successful implementation.

! Look for invisible monitoring opportunities, which impose virtually no burden on athletes. The data collection is automated, and nothing else is needed from the athletes. These invisible opportunities, whatever they may be, will still require data aggregation and integration from staff and require an environment for informed decision making, but they may enhance implementation success when athlete buy-in is the primary barrier.

! Build technological implementation into existing and new routines and tasks to make it obvious that technology is being integrated and something new is being introduced.

The Final Question: Is the Technology Worth It?

The final step, which should only be considered seriously once all 4 previous questions have been answered affirmatively, is whether the technology is worth the investment. This is essentially a cost-benefit analysis comparing the expected net performance effect with the financial burden that the technology carries in the overall context of the other 4 questions in the framework.

On the Flip Side: Delivering With Technology

Although our focus in this article was predominantly on guiding researchers and practitioners to critically evaluate whether they should purchase a new technology, we believe it would be a mistake not to briefly address several key considerations for when that technology is introduced.

The following concrete recommendations may facilitate buy-in and increase the probability that technological implementation succeeds: (1) undersell and overdeliver; (2) allow athletes to provide the equivalent of informed consent before data collection; (3) try to include multiple key stakeholders in data-evaluation sessions so that the technology does not isolate support staff or create a ‘‘secret society’’; (4) continue to evaluate signal-to-noise ratios and provide clarity on accuracy and reliability; (5) allow complex analyses to take place behind closed doors but present simplified, clean data that support important messages to coaches and athletes; and (6) do not use data in ways that contribute to political agendas or undermine the integrity of colleagues.

Integrating advanced technology into high-performance sport can be challenging. Emotions, egos, time constraints, and unrealistic expectations can make it difficult to gain approval as well as purchase and implement new technology. Many young practitioners may purchase

technology in an attempt to win favor from upper management as they work to encourage enthusiasm, hope, and belief in players and stakeholders. Without being fully aware of the power of the placebo (that is, the excitement of buying something new), they may convince sports administrators to allocate significant funds for a speculative purchase. We hope that the critical framework provided in this article will help to prevent these types of poor decisions, yet we believe the placebo, or belief, effect is an important aspect of technological implementation to leverage.

Interestingly, it has been reported that some of the earliest placebo researchers examined the influence of fake advanced medical technology for treating pain. For those who believed new technology could take away pain, the sophisticated device that supposedly harnessed the power of special metals worked equally well when the devices looked like they were made of metal but were actually made of wood.52 Introducing advanced technology into a high-performance program may have similarly positive effects if it is sold and implemented the right way, with a collective belief among practitioners and athletes that the technology can have a positive effect. In contrast, organizational differences of opinion can dilute the power of the belief effect. When the support team is divided on the efficacy of using new technology, their conversations and attitudes can ultimately undermine buy-in from athletes. Technological implementation should ultimately be a collective team effort whereby stakeholders engage throughout the stages of the decision-making framework and as the technology is implemented.

CONCLUSIONS

Technology may help organizations reach their grandiose vision or drag companies down because of its pitfalls. We hope this critical framework will empower practitioners and organizations to make informed, wise decisions about whether a technology should be implemented. However, we acknowledge that this stepwise approach is an oversimpli- fication and over-regimentation of the technological- evaluation process. At times, a technology may simply cost too much, in which case the 4 questions are irrelevant. Several questions may be investigated at once, such as when a company provides a free trial. Although the framework may be applied differently than described in this stepwise presentation, we caution that all 4 questions are critical to answer affirmatively before an investment is made.

Three fundamental principles underpin this type of a framework: (1) proactivity allows practitioners to start with the end in mind and to plan ahead in considering how to solve implementation challenges across the data pipeline; (2) critical thinking informs how practitioners evaluate the trustworthiness of technology and its data, as well as the intended and unintended consequences of introducing the technology; and (3) collaboration, specif- ically internal collaboration, underpins the success of communication and data-informed decisions and external collaboration can be essential for piloting technologies where appropriate (eg, outsourcing validity-related, data management, or analytical work that is beyond an organization’s current capabilities). Each of these princi-

908 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

ples bodes well in critically thinking about technology and even more broadly for practitioner and organizational excellence.

The decision ‘‘to tech or not to tech’’ is critical but complex. It should be made through a careful evaluation of evidence related to the technology and the environment in which it will be deployed. Ultimately, it is a question that every organization and practitioner will face in today’s technologically driven age. Much like the technologies in question, no person or company will make all perfect decisions, but a thoughtful framework and critical approach can help them hit the target more often than not.

REFERENCES

1. Bourdon PC, Cardinale M, Murray A, et al. Monitoring athlete

training loads: consensus statement. Int J Sports Physiol Perform.

2017;12(suppl 2):S2161–S2170. doi:10.1123/IJSPP.2017-0208

2. Borresen J, Lambert MI. The quantification of training load, the

training response and the effect on performance. Sports Med.

2012;39(9):779–795. doi:10.2165/11317780-000000000-00000

3. Saw AE, Main LC, Gastin PB. Monitoring the athlete training

response: subjective self-reported measures trump commonly used

objective measures: a systematic review. Br J Sports Med.

2016;50(5):281–291. doi:10.1136/bjsports-2015-094758

4. Neupert EC, Cotterill ST, Jobson SA. Training-monitoring engage-

ment: an evidence-based approach in elite sport. Int J Sports Physiol

Perform. 2018;14(1):99–104. doi:10.1123/ijspp.2018-0098

5. Nässi A, Ferrauti A, Meyer T, Pfeiffer M, Kellmann M.

Psychological tools used for monitoring training responses of

athletes. Perform Enhanc Health. 2017;5(4):125–133. doi:10.1016/

j.peh.2017.05.001

6. Flatt AA, Esco MR. Smartphone-derived heart-rate variability and

training load in a women’s soccer team. Int J Sports Physiol

Perform. 2015;10(8):994–1000. doi:10.1123/ijspp.2014-0556

7. Buchheit M. Monitoring training status with HR measures: do all

roads lead to Rome? Front Physiol. 2014;5:73. doi:10.3389/fphys.

2014.00073

8. Gathercole R, Sporer B, Stellingwerff T. Countermovement jump

performance with increased training loads in elite female rugby

athletes. Int J Sports Med. 2015;36(9):722–728. doi:10.1055/s-

0035-1547262

9. Wu PP-Y, Sterkenburg N, Everett K, Chapman DW, White N,

Mengersen K. Predicting fatigue using countermovement jump

force-time signatures: PCA can distinguish neuromuscular versus

metabolic fatigue. PLoS One. 2019;14(7):e0219295. doi:10.1371/

journal.pone.0219295

10. Meng E, Sheybani R. Insight: implantable medical devices. Lab

Chip. 2014;14(17):3233–3240. doi:10.1039/C4LC00127C

11. van der Kruk E, Reijne MM. Accuracy of human motion capture

systems for sport applications; state-of-the-art review. Eur J Sport

Sci. 2018;18(6):806–819. doi:10.1080/17461391.2018.1463397

12. Grigg J, Haakonssen E, Rathbone E, Orr R, Keogh JWL. The

validity and intra-tester reliability of markerless motion capture to

analyse kinematics of the BMX Supercross gate start. Sports

Biomech. 2018;17(3):383–401. doi:10.1080/14763141.2017.

1353129

13. Bruderer T, Gaisl T, Gaugg MT, et al. On-line analysis of exhaled

breath. Chem Rev. 2019;119(19):10803–10828. doi:10.1021/acs.

chemrev.9b00005

14. Peake JM, Kerr G, Sullivan JP. A critical review of consumer

wearables, mobile applications, and equipment for providing

biofeedback, monitoring stress, and sleep in physically active

populations. Front Physiol. 2018;9:743. doi:10.3389/fphys.2018.

00743

15. Petersen CJ, Pyne DB, Portus MR, Dawson BT. Comparison of

player movement patterns between 1-day and test cricket. J Strength

Cond Res. 2011;25(5):1368–1373.

16. Gray AJ, Jenkins DG. Match analysis and the physiological

demands of Australian football. Sports Med. 2010;40(4):347–360.

doi:10.2165/11531400-000000000-00000

17. Michalsik LB, Madsen K, Aagaard P. Match performance and

physiological capacity of female elite team handball players. Int J

Sports Med. 2014;35(7):595–607.

18. Bangsbo J, Nrregaard L, Thors F. Activity profile of competition

soccer. Can J Sport Sci. 1991;16(2):110–116.

19. Reilly T. Energetics of high-intensity exercise (soccer) with

particular reference to fatigue. J Sports Sci. 1997;15(3):257–263.

doi:10.1080/026404197367263

20. Reilly T, Thomas V. A motion analysis of work-rate in different

positional roles in professional football match-play. J Hum Mov

Stud. 1976;2:87–97.

21. Barris S, Button C. A review of vision-based motion analysis in

sport. Sports Med. 2008;38(12):1025–1043. doi:10.2165/00007256-

200838120-00006

22. Cummins C, Orr R, O’Connor H, West C. Global positioning

systems (GPS) and microtechnology sensors in team sports: a

systematic review. Sports Med. 2013;43(10):1025–1042. doi:10.

1007/s40279-013-0069-2

23. Torreño N, Munguı́a-Izquierdo D, Coutts A, de Villarreal ES,

Asian-Clemente J, Suarez-Arrones L. Relationship between external

and internal loads of professional soccer players during full-matches

in official games using global positioning systems and heart-rate

technology. Int J Sports Physiol Perform. 2016;11(7):940–946.

doi:10.1123/ijspp.2015-0252

24. Li RT, Kling SR, Salata MJ, Cupp SA, Sheehan J, Voos JE.

Wearable performance devices in sports medicine. Sports Health.

2016;8(1):74–78. doi:10.1177/1941738115616917

25. FIFA Quality Performance Reports for EPTS. Football Technology.

FIFA Web site. https://football-technology.fifa.com/en/media-tiles/

fifa-quality-performance-reports-for-epts/. Accessed February 4,

2020.

26. Bradley PS, Lago-Peñas C, Rey E, Gomez Diaz A. The effect of

high and low percentage ball possession on physical and technical

profiles in English FA Premier League soccer matches. J Sports Sci.

2013;31(12):1261–1270. doi:10.1080/02640414.2013.786185

27. Bush MD, Archer DT, Hogg R, Bradley PS. Factors influencing

physical and technical variability in the English Premier League. Int

J Sports Physiol Perform. 2015;10(7):865–872. doi:10.1123/ijspp.

2014-0484

28. Gregson W, Drust B, Atkinson G, Salvo VD. Match-to-match

variability of high-speed activities in premier league soccer. Int J

Sports Med. 2010;31(4):237–242. doi:10.1055/s-0030-1247546

29. MacDonald KJ, Palacios-Derflingher LM, Emery CA, Meeuwisse

WH. The effect of injury definition and surveillance methodology

on measures of injury occurrence and burden in elite volleyball. Int

J Sports Med. 2018;39(11):860–866. doi:10.1055/a-0577-4639

30. Bere T, Kruczynski J, Veintimilla N, Hamu Y, Bahr R. Injury risk is

low among world-class volleyball players: 4-year data from the

FIVB Injury Surveillance System. Br J Sports Med .

2015;49(17):1132–1137. doi:10.1136/bjsports-2015-094959

31. Helland C, Bojsen-Mller J, Raastad T, et al. Mechanical properties

of the patellar tendon in elite volleyball players with and without

patellar tendinopathy. Br J Sports Med. 2013;47(13):862–868.

doi:10.1136/bjsports-2013-092275

32. MacDonald K, Bahr R, Baltich J, Whittaker JL, Meeuwisse WH.

Validation of an inertial measurement unit for the measurement of

jump count and height. Phys Ther Sport. 2017;25:15–19. doi:10.

1016/j.ptsp.2016.12.001

33. Mujika I, Halson S, Burke LM, Balagué G, Farrow D. An

integrated, multifactorial approach to periodization for optimal

Journal of Athletic Training 909

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

performance in individual and team sports. Int J Sports Physiol

Perform. 2018;13(5):538–561. doi:10.1123/ijspp.2018-0093 34. Kiely J. Periodization theory: confronting an inconvenient truth.

Sports Med. 2018;48(4):753–764. doi:10.1007/s40279-017-0823-y 35. Haugen T, Buchheit M. Sprint running performance monitoring:

methodological and practical considerations. Sports Med. 2015;46(5):641–656. doi:10.1007/s40279-015-0446-0

36. Coutts AJ, Duffield R. Validity and reliability of GPS devices for measuring movement demands of team sports. J Sci Med Sport.

2010;13(1):133–135. doi:10.1016/j.jsams.2008.09.015 37. Bellenger CR, Fuller JT, Thomson RL, Davison K, Robertson EY,

Buckley JD. Monitoring athletic training status through autonomic heart rate regulation: a systematic review and meta-analysis. Sports

Med. 2016;46(10):1461–1486. doi:10.1007/s40279-016-0484-2

38. Kelly JM, Strecker RE, Bianchi MT. Recent developments in home sleep-monitoring devices. ISRN Neurol. 2012:768794. doi:10.5402/

2012/768794 39. Torres-Ronda L, Schelling X. critical process for the implementa-

tion of technology in sport organizations. Strength Cond J. 2017;39(6):54–59. doi:10.1519/SSC.0000000000000339

40. Liebermann DG, Katz L, Hughes MD, Bartlett RM, McClements J, Franks IM. Advances in the application of information technology

to sport performance. J Sports Sci. 2002;20(10):755–769. doi:10. 1080/026404102320675611

41. Loevinger J. Objective tests as instruments of psychological theory. Psychol Rep. 1957;3(3):635–694. doi:10.2466/pr0.1957.3.3.635

42. Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull. 1955;52(4):281–302.

43. Messick S. Foundations of validity: meaning and consequences in psychological assessment. ETS Res Rep Series. 1993;1993(2):i–18.

doi:10.1002/j.2333-8504.1993.tb01562.x

44. Messick S. Validity of test interpretation and use. ETS Res Rep

Series. 1990;1990(1):1487–1495. doi:10.1002/j.2333-8504.1990.

tb01343.x

45. Messick S. The standard problem: meaning and values in

measurement and evaluation. Am Psychol. 1975;30(10):955–966.

doi:10.1037/0003-066X.30.10.955

46. American Educational Research Association, American Psycholog-

ical Association, and National Council on Measurement in

Education. Standards for Educational and Psychological Testing.

Washington, DC: American Psychological Association; 1999.

47. Dayal U, Castellanos M, Simitsis A, Wilkinson K. Data integration

flows for business intelligence. In: Kersten M, Novikov B, Teubner

J, eds. EDBT ’09: Proceedings of the 12th International Conference

on Extending Database Technology: Advances in Database

Technology. Saint Petersburg, Russia; March 22, 2009:1–11.

https://dl.acm.org/doi/abs/10.1145/1516360.1516362. Accessed

May 20, 2020.

48. Bolger N, Laurenceau J-P. Intensive Longitudinal Methods: An

Introduction to Diary and Experience Sampling Research. New

York, NY: Guilford Press; 2013.

49. Walls TA, Schafer JL, eds. Models for Intensive Longitudinal Data.

New York, NY: Oxford University Press; 2006.

50. Windt J, Ardern CL, Gabbett TJ, et al. Getting the most out of

intensive longitudinal data: a methodological review of workload–

injury studies. BMJ Open. 2018;8(10):e022626. doi:10.1136/

bmjopen-2018-022626

51. Broman KW, Woo KH. Data organization in spreadsheets. Am Stat.

2018;72(1):2–10. doi:10.1080/00031305.2017.1375989

52. Finniss DG. Placebo effects: historical and modern evaluation. Int

Rev Neurobiol. 2018;139:1–27.

Address correspondence to Johann Windt, PhD, CSCS, Vancouver Whitecaps FC, 3065 Wesbrook Mall, Vancouver, BC V6T 1Z3, Canada. Address email to [email protected].

910 Volume 55 ! Number 9 ! September 2020

D ow

nloaded from http://m

eridian.allenpress.com /jat/article-pdf/55/9/902/2596927/i1062-6050-55-9-902.pdf by Florida Institute of Technology user on 30 N

ovem ber 2021

REVIEW ARTICLE OPEN

Wearable sensors for monitoring the internal and external workload of the athlete Dhruv R. Seshadri1, Ryan T. Li2, James E. Voos3, James R. Rowbottom4, Celeste M. Alfes5, Christian A. Zorman 6 and Colin K. Drummond1

The convergence of semiconductor technology, physiology, and predictive health analytics from wearable devices has advanced its clinical and translational utility for sports. The detection and subsequent application of metrics pertinent to and indicative of the physical performance, physiological status, biochemical composition, and mental alertness of the athlete has been shown to reduce the risk of injuries and improve performance and has enabled the development of athlete-centered protocols and treatment plans by team physicians and trainers. Our discussions in this review include commercially available devices, as well as those described in scientific literature to provide an understanding of wearable sensors for sports medicine. The primary objective of this paper is to provide a comprehensive review of the applications of wearable technology for assessing the biomechanical and physiological parameters of the athlete. A secondary objective of this paper is to identify collaborative research opportunities among academic research groups, sports medicine health clinics, and sports team performance programs to further the utility of this technology to assist in the return-to-play for athletes across various sporting domains. A companion paper discusses the use of wearables to monitor the biochemical profile and mental acuity of the athlete.

npj Digital Medicine (2019) 2:71 ; https://doi.org/10.1038/s41746-019-0149-2

INTRODUCTION Technological advancements have enabled athletes, coaches, and physicians to track functional movements, workload, biomecha- nical and bio-vital markers utilizing wearable sensors to maximize performance and minimize the potential for injury.1–3 Wearable monitoring systems can provide continuous physiological data thus permitting the development of accurate treatment plans and player-specific training programs to potentially mitigate and alleviate injuries.4 Herein, we define a wearable device as a sensor or sensor suite unencumbered by wires for the continuous and non-invasive detection of biosignals, analytes, or biomechanical and impact forces for monitoring human health and performance. Over the past two decades, the wearables field has moved from a device to a systems viewpoint, where the system combines the device with analytics. While previous literature has reviewed specific technical domains of the wearables field, such as sensors,5–8 materials,9–12

and soft interfaces13–15 or focused on the fabrication and application of such devices to address a specific medical condition, such as atrial fibrillation,16–18 cystic fibrosis,19–21 or diabetes,22–27

there remains an unmet medical need to assess, develop, and validate this technology specifically for sports medicine. Given the heightened attention to athlete safety and performance, this review evaluates the translational utility of wearable devices to detect key metrics pertinent to human performance assessment (Fig. 1). The organization of this review is structured around discussing the

value wearable sensors provide in sports to monitor player activity levels and mitigate injury. We progress through this review by discussing the utility of wearable sensors in two domains crucial to

human performance ranging from an athlete’s physical performance and physiological status. Our goal in each of these areas is centered around reviewing both scientific literature and current commercially available devices to provide a comprehensive view of wearables for sports medicine (Tables 1–5). This review is specifically targeted towards those whose interests lie in the application and translation of wearable sensors for assessing human performance.

PHYSICAL PERFORMANCE AND SAFETY OF THE ATHLETE Position and motion The ability to monitor position and movement profiles of an athlete is critical in developing improved training regimens to maximize individual performance (Fig. 2). The accuracy of devices, such as pedometers has been in question and was recently studied. Researchers compared the accuracy of the “step-count” feature between dedicated smartphone-based pedometer appli- cations (Galaxy S4 Moves App, iPhone 5s Moves App, iPhone 5s Health Mate App, iPhone 5s Fitbit App) and wearable devices (Nike Fuelband, Jawbone UP24, Fitbit Flex, Fitbit One, Fitbit Zip, and Digi-Walker SW-200) with direct observation of step counts.28

Results showed a relative difference between actual and reported mean step count of −0.3% to 1.0% for pedometers and accelerometers, −22.7% to −1.5% for wearable devices, and −6.7% to 6.2% for smartphone applications. Such differences were attributed to the robustness of the IC technology and software algorithms used to determine a step. Step counts are often used to derive other measures of physical activity, such as distance

Received: 19 January 2018 Accepted: 8 July 2019

1Department of Biomedical Engineering, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106, USA; 2Department of Orthopaedic Surgery, University Hospitals Cleveland Medical Center, Cleveland, OH 44106, USA; 3University Hospitals Sports Medicine Institute, Cleveland, OH 44106, USA; 4Department of Cardiothoracic Anesthesiology, The Cleveland Clinic, 9500 Euclid Avenue, Cleveland, OH 44195, USA; 5Frances Payne Bolton School of Nursing, Case Western Reserve University, 9501 Euclid Avenue, Cleveland, OH 44106, USA and 6Department of Electrical Engineering and Computer Science, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106, USA Correspondence: Dhruv R. Seshadri ([email protected])

www.nature.com/npjdigitalmed

Scripps Research Translational Institute

traveled or calories expended.28 Hence, improving measurement accuracy is crucial to measure and appropriately tailor workout regiments for elite-level athletes. Movement-based sensors currently in use for sports-medicine

include accelerometers and global positioning satellite (GPS) devices, often used in combination (Table 1). Accelerometers generate highly accurate analyses of movement with high sampling rates and have been included in wrist-based devices, such as the Nike Fuelband, Jawbone UP, and Microsoft Band. This technology has been widely adopted in the sporting community ranging from Australian Football,29 Rugby,30,31 NFL,32 National Hockey League (NHL),33 and swimming.34–36 Energy expenditure can be determined from tri-axial accelerometers via the integration of acceleration over time.37,38 The determination of energy expenditure, position, movement, and balance control during practices or games has shown to be instrumental in tailoring the training regimen of athletes to minimize the incidence of soft tissue injuries. Banister et al. postulated that athletic performance can be

estimated as a function of fatigue and fitness39 (Fig. 2). Building upon this model, Morton et al. suggested that an opportune training stimulus is one that maximizes performance by utilizing an appropriate training load, while simultaneously minimizing injury and fatigue.40 A working definition of fatigue is “any exercise-induced or non-exercise-induced loss in total perfor- mance due to various physiological factors, athlete reported psychological factors, or a combination of the two”.41 It is well known that fatigue decreases athletic performance and that training induces numerous neurophysiological and psychological changes in an athlete’s body. There are two forms of fatigue: central fatigue and peripheral fatigue. Central fatigue is the fatigue resulting from the central nervous system (CNS) and the transmission of signals from the brain to the muscle.42 Central fatigue is related to the interaction between the brain and the spinal cord.43 Researchers have hypothesized that the differentia- tion between a good athlete and an elite-level athlete is their individual ability to ignore such neurotransmissions during high-

acuity situations (e.g. high profile matches or workouts).42

Peripheral fatigue is the failure to maintain an expected power output caused by the depletion of glycogen, phosphate compounds, or acetylcholine within the muscular unit or by the accumulation of lactate or other metabolites that are released during activity.44,45 Peripheral fatigue occurs within the muscle and can be thought of as ‘muscle fatigue’.43 As such, wearable sensors can be used to measure parameters indicative of the peripheral fatigue of the athlete, as is discussed in detail throughout this review. For simplicity purposes, we refer to peripheral fatigue as simply fatigue. Monitoring internal (e.g. physiological or perceptual ‘response’)

and external training loads (e.g. physical ‘work’) can enable sports trainers and clinicians to assess the fatigue and fitness levels of athletes in real time. Internal workload includes the session rate of perceived exertion (sRPE) and heart rate.46 At the completion of each training session, athletes provide a 1–10 ‘rating’ based on the intensity of the session.46 The intensity of the session is multiplied by the session duration to provide the internal training load.46 The product can be thought of as the athletes’ “exertional minutes”.46

Advancements in MEMS fabrication techniques and device packaging have allowed for the detection of multi-axial move- ment to calculate an external training load (e.g. PlayerLoad™3). External workload can be thought of as how much load is placed on the body and can be quantified using torso-mounted wearable devices which contain a GPS and a tri-axial accelerometer.46

PlayerLoad™ can be calculated via the instantaneous rate of change of acceleration. Accumulated PlayerLoad™ can be calculated as the summation of PlayerLoad™ over the desired time interval (usually over a span of 1–7 days). Metrics such as total distance run, weight lifted, number and

intensity of sprints or collisions can be determined using GPS- based sensors. Position sensors triangulate signal transmission from multiple GPS satellites orbiting the earth and can accurately determine the velocity and position (within 1 m) of an athlete on a field. These devices are playing an instrumental role in sports performance analysis by allowing coaches, physicians, and trainers to better understand real-time physical demands of an ath- lete.30,37,47 GPS silicon chips combined with tri-axial acceler- ometers have been used to record physical activities during different times of the day and for specific position groups on a team.48 The majority of work to assess human motion and its correlation to sports performance has involved the use of commercial GPS-based devices, such as the Catapult devices (OptimEye S5) and Zebra Technologies GPS device. The Catapult product, for example has a fully packaged processing IC, accelerometer, gyroscope, and magnetometer to measure body position, impact forces, velocity, acceleration, and direction in a continuous manner.49 In a study utilizing the Catapult OptimEye S5 and video tracking technology, 20 professional Australian Football League (AFL) players were studied during four in-season matches to describe and quantify the frequency, velocity, and acceleration at impact during tackling29 (Fig. 3a–c). Distributions in tackles were quantified and classified as a function of percent distribution of tackles versus player load (Fig. 3a), player velocity versus tackle intensity (Fig. 3b), and player load versus tackle intensity (Fig. 3c). Differences in accelerometer data between tackles were observed to be progressively greater in intensity thereby providing support for the use of accelerometers to assess impact forces in contact-based sports.29 In another study, GPS sensors and related analytics were used by National Collegiate Athletic Association (NCAA) Division I Football athletes to record workload, velocity, distance, and acceleration during both practices and games.48,50 The studies found significant variation in movement profiles among collegiate football players and the authors identified the need for position-specific and game-specific physical conditioning strategies to maximize player performance, limit the effects of fatigue, and minimize the onset of injuries.

Fig. 1 Four areas of focus as it relates to assessing human performance. The central theme of this review is the use of wearable sensors to maximize the performance and safety of the athlete. This involves the detection and measurement of the internal and external workload of the athlete which are based on the athlete’s physical performance, physiological status, biochemical composition, and mental acuity

D.R. Seshadri et al.

2

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

1 2 3 4 5 6 7 8 9 0 () :,;

Ta bl e 1.

Ex am

p le s o f w ea ra b le

te ch

n o lo g y co

m p an

ie s w it h p ro d u ct s ap

p lic ab

le to w ar d s as se ss in g th e p o si ti o n an

d m o ti o n o f th e at h le te

C o m p an

y Sa m p lin

g o f p ro d u ct s

Pr o d u ct

ty p e

Pr o d u ct

fu n ct io n al it y

H ea d q u ar te rs

A d id as

m iC o ac h Fi t Sm

ar t, m iC o ac h

Sm ar t R u n

W at ch

H ea rt

ra te ,G

PS ,d

is ta n ce

H er zo g en

au ra ch

,G er m an

y

A p p le

A p p le

W at ch

W at ch

H ea rt

ra te ,d

is ta n ce ,e

m ai l, EC

G ,t ex t m es sa g es ,p

h o n e

C u p er ti n o,

C A

B io Se

n si ve

Te ch

n o lo g ie s In c.

Jo u le

Ea rr in g s

H ea rt

ra te ,c al o ri es

b u rn ed

,s te p s ta ke n ,o

ve ra ll ac ti vi ty

le ve l

O n ta ri o,

C an

ad a

C at ap

u lt

O p ti m Ey e S5

,V ec to r

D ev

ic e u n it

M o ve m en

t, Tu

rn ra te s, o ri en

ta ti o n ,h

ea rt ra te .D

ev ic e p la ce d b el o w

th e n ec k (t u ck ed

in sh o u ld er

p ad

s) M el b o u rn e,

A u st ra lia

Fi tb it

Fl ex ,O

n e,

A lt a

W at ch

St ep

s w al ke d ,d

is ta n ce ,h

ea rt

ra te ,s le ep

q u al it y, p ed

o m et er ,

ca lo ri es

b u rn ed

Sa n Fr an

ci sc o,

C A

G ar m in

V iv o ac ti ve

,V iv o sm

ar t, V iv o fi t

W at ch

Pe d o m et er ,s le ep

q u al it y, h ea rt

ra te ,d

is ta n ce

Sc h af fh au

se n ,S

w it ze rl an

d

Ja b ra

Sp o rt s Pu

ls e W ir el es s

H ea d p h o n e

H ea d p h o n e

A cc el er o m et er

an d h ea rt

ra te

m o n it o ri n g

B al le ru p ,D

en m ar k

Ja w b o n e

U p

B an

d Pe

d o m et er ,d

is ta n ce ,h

ea rt

ra te ,s le ep

q u al it y, ca lo ri es

Sa n Fr an

ci sc o,

C A

K ar ac u s

Po la ri s, Z et a,

Pr o xi m a

W at ch

M o ve m en

t, p h o n e,

em ai l

C h ap

el H ill ,N

C

K it m an

La b s

C ap

tu re

Se n so r m o u n te d o n

co m p u te r

B io m et ri c d at a vi a m ac h in e le ar n in g ,G

PS ,a

n d p la ye r tr ac ki n g

D u b lin

,I re la n d

M ic ro so ft

M ic ro so ft B an

d B an

d H ea rt

ra te ,c al o ri es

b u rn ed

,s le ep

q u al it y, em

ai l, te xt

R ed

m o n d ,W

A

N ik e

Fu el b an

d B an

d Pe

d o m et er ,G

PS W as h in g to n C o u n ty ,O

re g o n

Po la r

A 36

0, Lo

o p C ry st al ,L

o o p 2

B an

d H ea rt

ra te ,p

er fo rm

an ce

tr ac ke r

K em

p el e,

Fi n la n d

Sa m su n g

G ea rF it 2

W at ch

G PS

,s le ep

,h ea rt

ra te ,c al o ri es ,p

ed o m et er

Se o u l, So

u th

K o re a

Sa n si b le

Te ch

n o lo g ie s

Li ve Sk in

Te xt ile

el ec tr o n ic s

Sp ee

d ,i m p ac t, p o si ti o n

Ed in b u rg h ,U

n it ed

K in g d o m

St ar ke y H ea ri n g Te ch

n o lo g ie s

Li vi o A I H ea ri n g A id

H ea ri n g ai d

Tr an

sl at es

fo re ig n la n g u ag

es ,c o n ta in s a p ed

o m et er ,t ra ck s p h ys ic al

ac ti vi ty

(w el ln es s sc o re )

Ed en

Pr ai ri e,

M N

St ifi t

St ifi t b an

d B an

d B lo o d o xy g en

,b o d y m as s in d ex ,c al o ri es

b u rn ed

,d is ta n ce ,f at ig u e,

h ea rt

ra te ,s le ep

Sa n Fr an

ci sc o,

C A

Tr u So

x Tr u So

x So

ck s

N o n -s lip

so ck s to

g en

er at e g re at er

sp ee

d an

d ag

ili ty

B al ti m o re ,M

D

U n d er

A rm

o r

H TC

G ri p

W ri st b an

d H ea rt

ra te ,c al o ri es

b u rn ed

,d is ta n ce

tr av el ed

B al ti m o re ,M

D

Ve rt

G -V er t, V ER

T C o ac h

D ev

ic e u n it

M ea su re

G -f o rc e in

m o ve

m en

t, ac ce le ra ti o n ,k in et ic

en er g y, p o w er

Fo rt

La u d er d al e,

FL

V ib ra d o Te ch

n o lo g ie s

V ib ra d o Te ch

n o lo g ie s

Te xt ile

el ec tr o n ic s

A cc el er o m et er

to m ea su re

sh o t an

g le ,a

rm h ei g h t, re le as e p o in t.

Sl ee

ve to

b e w o rn

o ve

r fo re ar m

Su n n yv al e,

C A

Z eb

ra Te ch

n o lo g ie s

Z eb

ra Tr ac ki n g D ev

ic e

D ev

ic e u n it

R FI D

u se d to

q u an

ti fy

m o ve

m en

t an

d d is ta n ce

p ro fi le s. D ev

ic e

p la ce d b el o w

th e n ec k in

sh o u ld er

p ad

s o r se w n in to

je rs ey

Li n co

ln sh ir e,

IL

Z ep

h yr

B io H ar n es s 3,

H xM

™ Sm

ar t,

H xM

™ BT

Te xt ile

El ec tr o n ic s

H ea rt

ra te ,r es p ir at io n ,t ri -a xi al

ac ce le ro m et er ,h

ea rt

ra te ,a

ct iv it y,

p o st u re ,o

xy g en

sa tu ra ti o n le ve ls

A n n ap

o lis ,M

ar yl an

d (F o u n d ed

in N ew

Z ea la n d )

D at a fo r th is ta b le

w as

ac q u ir ed

fr o m

co m p an

y w eb

si te s an

d so ci al

m ed

ia si te s af fi lia te d w it h ea ch

co m p an

y

D.R. Seshadri et al.

3

Scripps Research Translational Institute npj Digital Medicine (2019) 71

The combination of the internal and external workloads of the athlete determine the training outcome.46 An athlete’s internal or external workload can be computed over a 1-week period (acute workload) and over a 3–4-week period (chronic workload). Work by Gabbett suggested that the ratio of the acute-to-chronic workload, herein referred to as “ACWR”, can be used to determine if an athlete is overtraining, undertraining, or training at the opportune intensity46 (Fig. 2b). Furthermore, Gabbett showed that calculation of this ratio enables sports scientists to predict the chance an athlete suffers an injury as a result of improper load management.46 Deriving this ratio provides an index of athlete preparedness and considers the training load that the athlete has performed relative to the training load that the athlete has been prepared for.51 The use of the ACWR emphasizes both the positive and negative consequences of training. The first study to investigate the relationship between ACWR and the risk of injury was performed on elite cricket fast bowlers.52 Training loads were estimated from both sRPE and balls bowled. When the acute workload was similar to or lower than that of the chronic workload (e.g. ACWR ≤ 0.99), the likelihood of injury for fast bowlers in the next 7 days was 4%.52 However, when the ACWR was ≥1.5 (e.g. workload in the current week was 1.5 times greater than what the bowler was prepared for), the risk of injury was 2–4 times greater

in the subsequent 7 days.52 While such observations are indicative of the sport being studied, until more robust data sets are available, caution must be heeded when applying these recommendations to individual sport athletes. Despite this, a general trend can be concluded. If the acute training load is low (e.g. the athlete is experiencing minimal fatigue) and the rolling average (RA) chronic training load is high (e.g. the athlete has developed ‘fitness’), then the athlete will be in a well-prepared state and thus, the ACWR will be ≤1.46 If the acute load is high (e.g. training loads have been rapidly increased resulting in ‘fatigue’) and the RA chronic training load is low (e.g. the athlete has performed inadequate training to develop ‘fitness’), then the athlete will be in a fatigued state and thus, the ACWR will be ≥1.46

In terms of injury risk, ACWRs within the range of 0.8–1.3 could be considered the training ‘sweet spot’, while an ACWR ≥ 1.5 could represent the ‘danger zone’.46

The RA model53 (Eqs. (1–4)) and exponentially weighted moving average (EWMA) model54 (Eqs. (5–10)) are two methods used to calculate the training load of the athlete with or without the use of wearable sensors like the Catapult OptimEye S5 (Eqs. (11–14)).3

The RA model uses an absolute (i.e. total) workload performed in one week (acute workload) relative to the 4-week chronic workload (e.g. 4-week average acute workload).53 Equation (1)

Table 2. Examples of wearable technology companies for impact monitoring

Company Sampling of products Product type Product functionality Headquarters

2ND Skull Cap, Band Garment Polyurethane-based composite dissipates impact Pittsburgh, PA

Athlete Intelligence Vector Mouthguard, Shockbox® sensor

Mouth guard Tracks linear and rotational accelerations of head impacts

Kirkland, WA

BrainScope Ahead 300 Hand-held point of care device

Disposable electrode sensors to detect head injuries

Bethesda, MD

Force Impact Technologies

Fitguard™ Mouth guard Embedded sensors relate collision intensity via color coded LED’s on the front of the mouth guard

Los Angeles, CA

Tempe, AZ

Hiji Hiji Band Head band Impact forces, intensity Phoenix, AZ

Jolt Jolt Sensor Sensor Impact forces, Concussion monitoring. Sensor clipped to garment

Boston, MA

Mamori Mamori Mouth guard Inertial sensors measure impact forces on the head

Dublin, Ireland

Noggin Pro Noggin, Noggin Pro Skull caps Gel capsules in skull cap dissipate forces from skull

Toronto, ON

Performance Sports Group

Q-Collar Neck collar Concussion prevention by applying pressure on the jugular vein

Cincinnati, OH

X2 Biosystems X-Patch Pro Flexible sensor Tri-axial accelerometers to measure impact Seattle, WA

X2 Mouthguard

Data for this table was acquired from company websites and social media sites affiliated with each company

Table 3. Examples of wearable technology companies for monitoring the biomechanical forces on the athlete

Company Sampling of products Product type Product functionality Headquarters

CricFlex CricFlex Sleeve Measures arm angle and force during bowling Islamabad, Pakistan

Heddoko Heddoko Smart Garment

Biomechanics of movement, deviation from benchmarks and movement standards, injury risk

Montreal, Canada

Motus Global mThrow™, motusPro™ Sleeve Accelerometer to measure joint angles, velocity, stress, strain

Massapequa, New York Ft. Lauderdale, FL

Protonics Technologies Protonics T2 Device Offsets left-right biomechanical imbalance to reduce muscle pain. Attached to left leg

Lincoln, NE

Data for this table was acquired from company websites and social media sites affiliated with each company

D.R. Seshadri et al.

4

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

Ta bl e 4.

Ex am

p le s o f w ea ra b le

te ch

n o lo g y co

m p an

ie s w it h p ro d u ct s ap

p lic ab

le to w ar d s m o n it o ri n g h ea rt

ra te

an d m u sc le

o xy g en

sa tu ra ti o n

C o m p an

y Sa m p lin

g o f p ro d u ct s

Pr o d u ct

ty p e

Pr o d u ct

fu n ct io n al it y

H ea d q u ar te rs

1s t R o u n d A th le ti cs

En er g yD

N A ™

B o d y su it

C o n ve

rt s h ea t to

IR w h ic h ex p an

d s b lo o d ve

ss el s fo r g re at er

b lo o d fl o w

Lo s A n g el es ,C

A

A th o s

A th o s W ea ra b le s

Ve st

an d p an

t M u sc le

ac ti vi ty

an d h ea rt

si g n al s vi a EM

G Sa n Fr an

ci sc o,

C A

H ex o Sk in

A st ro sk in , Sm

ar t K it

Sl ee

ve C ar d ia c fr eq

u en

cy ,r es p ir at o ry

ra te

an d vo

lu m e,

sl ee

p ,a

cc el er at io n

M o n tr ea l, C an

ad a

Sa n Fr an

ci sc o,

C A

H u aw

ei H o n o r B an

d A 1

W at ch

C ar d io -r es p ir at o ry

fi tn es s

Sh en

ze n ,C

h in a

H u m o n

H ex

D ev

ic e u n it

N o n -in

va si ve

m ea su re m en

t o f O 2 co

n te n t in

m u sc le s

B o st o n ,M

A

K o m o d o Te ch

n o lo g ie s In c.

A IO

Sm ar t Sl ee

ve Sl ee

ve EC

G ,h

ea rt

ra te , sl ee

p an

al ys is

W in n ip eg

,C an

ad a

Ky m ir a

Ky m ir a Sp

o rt s

T- Sh

ir t

Sm ar t g ar m en

ts fo r ca rd ia c m o n it o ri n g to

p re ve n t h ea rt at ta ck s in

at h le te s

R ea d in g ,U

n it ed

K in g d o m

Li fe B ea m

Li fe B ea m

b as eb

al l ca p

Sm ar tH at

Em b ed

d ed

se n so rs

m ea su re

h ea rt

ra te

an d ca lo ri es

N ew

Yo rk ,N

Y

M C 10

B io St am

p R C ™ ,B

io St am

p n Po

in t® ,

K in ti n u u m

Ep id er m al

se n so r

B io St am

p R C ™ :a

ct iv it y, ca rd ia c ac ti vi ty ,E

M G ,a

n d p o st u re

B o st o n ,M

A

B io St am

p n Po

in t® :a

ct iv it y, p o st u re

EM G ,a

n d sl ee

p m et ri c, vi ta l si g n s

K in ti n u u m :q

u an

ti fy

tr ea tm

en t ef fi ca cy

M yo

vo lt

M yo

vo lt

Sl ee

ve EM

G to

in cr ea se

ci rc u la ti o n ,b

o o st

m u sc le

p o w er

H o n g K o n g,

C h in a

R o te x Te ch

n o lo g ie s

ro M ag

e, ro Sp

o rt , ro C ar e,

ro Fa sh io n

El ec tr o n ic

ta tt o o

ro M ag

e: b ra in

an d m u sc le

co n tr o l, g es tu re

re co

g n it io n

A u st in ,T

X

ro Sp

o rt :b

lo o d O 2 sa tu ra ti o n ,h

ea rt

ra te , sk in

h yd

ra ti o n

ro C ar e:

b lo o d p re ss u re ,E

C G ,r es p ir at io n ,s ki n h yd

ra ti o n ,t em

p er at u re

ro Fa sh io n :g

lo w s o n sk in

(f as h io n p u rp o se s)

Sp ir e

Sp ir e

D ev

ic e u n it

M ea su re s re sp ir at io n to

q u an

ti fy

an d d et ec t st re ss ,c al o ri e tr ac ki n g ,

p ed

o m et er

Sa n Fr an

ci sc o,

C A

W it h in g s

St ee

l H R ,A

ct iv it é,

G o,

Pu ls e O 2

B an

d s

H ea rt

ra te ,d

is ta n ce s, em

ai l, te xt

m es sa g es , p h o n e

Is sy -le

s- M o u lin

ea u x,

Fr an

ce

X ia o m i

M i B an

d B an

d Ti m e,

p ed

o m et er ,h

ea rt

ra te

B ei jin

g ,C

h in a

X m et ri cs

X m et ri cs

Pr o,

X m et ri cs

Fi t

D ev

ic e u n it

C al o ri es ,s tr o ke s ta ke n ,l ap

s sw

am ,p

ac e

M ila n , It al y

D at a fo r th is ta b le

w as

ac q u ir ed

fr o m

co m p an

y w eb

si te s an

d so ci al

m ed

ia si te s af fi lia te d w it h ea ch

co m p an

y

D.R. Seshadri et al.

5

Scripps Research Translational Institute npj Digital Medicine (2019) 71

represents the exertional minutes per workout which is the product of the session rating of perceived exertion and the duration of the workout in minutes. The sRPE is a scale from 1 to 10 with progressing intensity of the workout deemed by the athlete and training staff. Equation (2) shows the acute player load (PL) which is the summation of the exertional minutes per workout for a given week (e.g. from day 1 to day 7). For the sake of simplicity, we assume the athlete is completing one workout per day. Equation (3) shows the chronic PL which is computed by taking the average of the acute PL over the duration of weeks (denoted as n). Equation (4) shows the ACWR which is the ratio between the acute PL for the given week (Eq. (2)) and the chronic PL (calculated from Eq. (3)). The RA model suggests that each workload in an acute and chronic period is equal. In other words, the model considers there to be a linear relationship between load and injury. The assumption in this model is that all workload in a

given time period is equivalent. Key drawbacks of this model are that the model does not account for any decays in fitness and it does not accurately represent variations in the manner in which loads are accumulated.

Exertional minutes per workout ¼ SRPEð Þ ´ duration of workout in minutesð Þ (1)

Acute PL ¼ XD¼7

D¼1 exertional minutes per workout (2)

Chronic PL ¼ PW¼n

W¼1 Acute PL n

(3)

ACWR ¼ Acute PL for given week Chronic PL

(4)

Table 5. Examples of wearable technology companies for monitoring sleep

Company Sampling of products Product type Product functionality Headquarters

Emfit Emfit QS Device unit Tracks sleep by monitoring movement and heart rate Vaajakoski, Finland

Kokoon Kokoon EEG Headphones Movement and EEG sensors determine relaxation and sleep quality Limerick, Ireland

Moov Moov Wrist-based device Heart rate, sleep quality, and activity tracker San Francisco, CA

WHOOP WHOOP Band Wrist-based device Heart rate, body temperature, movement, and sleep Boston, MA

Data for this table was acquired from company websites and social media sites affiliated with each company

Fig. 2 Value proposition of wearable sensor technology to monitor athlete training load to minimize soft-tissue injuries. a Hypothetical relationship between training loads, fitness, injuries, and performance. Inadequate and excessive training loads could result in increased injuries, reduced fitness, and poor team performance. b Interpreting and applying ACWR data to predict the likelihood of subsequent injury. The green-shaded area (‘sweet spot’) represents the ACWR where the risk of injury is low. The red-shaded area (‘danger zone’) represents the ACWR where the risk of injury is high. To minimize the risk of injury, athletes should aim to maintain their ACWR within a range of ~0.8–1.3. c Athlete workout can be monitored via workout logs and self-tracking methods, assessing the sRPE levels, or using wearable technology to quantify movement parameters. The application of wearable sensors to monitor athletic performance and training has provided an added advantage compared to current and past methods by enabling sports scientists and clinicians to quantify the workout, to calculate the ACWR, and to predict the onset of injury. Figure was adapted and modified from Gabbett et al. 46 a, b

D.R. Seshadri et al.

6

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

Sports scientists have started to apply the EWMA model to circumvent the drawbacks posed by the RA model.54 The EWMA model places a greater weight on the most recent workload an athlete has performed by assigning a decreasing weighting for each older workload value (time decay constant, λa) and the non- linear nature of injury occurrence and workload.54 Equation (5) shows the exertional minutes per workout which is the product of the session rating of perceived exertion and the duration of the workout in minutes. The sRPE is a scale from 1 to 10 with progressing intensity of the workout deemed by the athlete and training staff. Equation (6) shows the degree of decay, λa, which is a value between 0 and 1, with higher values of λa discounting older observations in the model at a faster rate. In the following equation, n represents the time decay constant. Equation (7) shows the formula to calculate the EWMA for a given day which is based on the exertional minutes, calculated from Eq. (5), the degree of decay from Eq. (6), and the EWMA from the preceding day. Equation (8) shows the acute player load (PL) which is the summation of the EWMA for a given week (e.g. from day 1 to day 7). For the sake of simplicity, we assume the athlete is completing 1 workout per day. Equation (9) shows the chronic PL which is

computed by taking the average of the acute PL over the duration of weeks (denoted as n). Equation (10) shows the ACWR which is the ratio between the acute PL for the given week (Eq. (8)) and the chronic PL (calculated from Eq. (9)). A recent study sought to investigate if any differences existed between the RA and EWMA models pertaining to ACWR calculation and subsequent injury risk in elite Australian footballers.54 Fifty-nine athletes from an AFL club participated in this 2-year study. A total of 92 individual sessions were recorded. Each season consisted of a 16-week preseason phase comprised of both running and football-based sessions, followed by a subsequent 23-week in-season competi- tive phase. The Catapult OptimEye S5 GPS sensor, sampled at 10 Hz, was used to quantify training and match workloads of players. The triaxial accelerometer, gyroscope, and magnetometer were each sampled at 100 Hz. The study demonstrated that a high ACWR was significantly associated with an increase in injury risk for both models. The EWMA model had significantly greater sensitivity to detect increases in injury likelihood at higher ACWR ranges during both the preseason and in-season periods. The study concluded that the EWMA model may be better suited to modeling workloads and injury risk than the RAs than the ACWR

Fig. 3 Wearable sensors monitor the biomechanical performance of the athlete. a Distribution of tackles (n= 352) made and against peak instantaneous Player Load™. b Peak velocity for tackles made and against associated with tackle intensity categorized as low (n= 115), medium (n= 216), and high (n= 21). c Peak Player Load™ for tackles made and against associated with tackle intensity categorized as low (n = 115), medium (n= 216), and high (n= 21). d Relative displacements of the mouthguard sensor from the skull studied using high speed video. Among 16 trials, the mouthguard always had the smallest (sub-millimeter) displacement from the skull, within video error, compared to the skull cap and skin patch. e Relative displacements of the Reebok skull cap from the skull studied using high speed video. f Relative displacements of the xPatch Gen2 skin patch sensor from the skull studied using high speed video. gmotusBASEBALL sensor exhibited higher peak elbow valgus torque for baseball pitching compared to football throwing. Data demonstrates the utility of the sensor to measure biomechanical forces during non-stationary periods on an athlete. h motusBASEBALL sensor used to quantify the average valgus torque on the elbow for baseball pitching and football throwing between foot contact and maximum internal rotation. “aSignificantly different (p < 0.01) from Low; bsignificantly different (p < 0.01) from Medium. No significant differences between tackles made and against.” Figures were reproduced with permission from Gastin et al.29 a–c, Wu et al.79 d–f, and Laughlin et al.89 g–h

D.R. Seshadri et al.

7

Scripps Research Translational Institute npj Digital Medicine (2019) 71

model. Regardless of the ACWR model utilized, spikes in acute workload were significantly associated with an increase in injury risk.

Exertional minutes per workout ¼ SRPEð Þ ´ duration of workout in minutesð Þ (5)

λa ¼ 2 N þ 1

;where 0<λa<1 (6)

EWMAtoday ¼ Exertional minutes per workoutð Þ λað Þ þ 1� λað Þ EWMAyesterday � �� �

(7)

Acute PL ¼ XD¼7

D¼1 EWMAtoday (8)

Chronic PL ¼ PW¼n

W¼1 Acute PL n

(9)

ACWR ¼ Acute PL for given week Chronic PL

(10)

Wearable sensors are currently being used to minimize injury in professional football via careful monitoring of training load and other biometrics during the rehabilitation period (Fig. 2c). The variability of GPS data and accelerations of the torso have been in question when it comes to monitoring the loads of the lower limbs. This is because distance traveled and velocity do not represent the mechanical load experienced by the musculoske- letal tissue. This is specifically relevant in sports such as basketball, which are constrained to a small-space, where players experience high loads of physical stress by performing explosive jumping and landing activities, which are not accurately captured by distance, speed, or torso athlete movement analysis systems.55,56 To mitigate such issues, the Zebra GPS device and Catapult OptimEye S5, both of which are considered the most accurate wearable devices in sports today, are housed in player tracking devices in an attempt to negate some of the aforementioned issues. Addition- ally, the Catapult device has shown to mitigate such issues by incorporating tri-axial movements into their analytical models to accurately calculate PlayerLoad™ from their sensor.3 The Zebra GPS device is currently approved by the NFL for use to track player movement and has been utilized by select teams to monitor training loads.57 Equation (11) provides the analytical platform of the Catapult OptimEye S5 which utilizes a tri-axial accelerometer to calculate PlayerLoad™ (PL) based on acceleration in the x, y, and z directions. Equation (12) shows the summation of PL from the initial to end time of interest (in most cases this is from the start to the end of a training session) denoted as AccPL™. Equation (13) shows how the RA model can be used to calculate Acute PL, analogous to Eq. (2). However, in this case, PL is calculated from Eq. (11) using a wearable sensor. Eq. (14) shows how the EWMA model can be used to calculate PL for the given day using PL calculated from a wearable sensor. The ACWR can be calculated utilizing either model, adapting the set of equations presented (rolling average, Eqs. (1–4); EWMA, Eqs. (5–10).

PL ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi fwdt¼iþ1 � fwdt¼ið Þ2þ sidet¼iþ1 � sidet¼ið Þ2þ upt¼iþ1 � upt¼i

� �2q

(11)

AccPL ¼ Xt¼n

t¼0

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi fwdt¼iþ1 � fwdt¼ið Þ2þ sidet¼iþ1 � sidet¼ið Þ2þ upt¼iþ1 � upt¼i

� �2q

(12)

Acute PL ¼ XD¼7

D¼1 AccPL (13)

EWMAtoday ¼ PLð Þ λað Þ þ 1� λað Þ EWMAyesterday � �� �

(14)

In a specific example reported by an American sporting network, the device was used to accurately track the recovery of

an athlete after the individual suffered a season ending injury the previous year.57 The sensor was placed underneath the shoulder pads (analogous to that of the Catapult device) or sewn into the jersey to generate biometric measurements, such as movement profiles and workload to gauge the athlete’s performance and workload during recovery relative to his peak performance and workload prior to the injury. Additionally, utilizing the Catapult OptimEye S5 wearable sensor, authors of this review have recently studied the effects of player workload on soft tissue injuries in a single NFL team over two seasons.32 Rapid changes in workload over a one-week period when compared to the average workload over a month were associated with a significant increase in risk of hamstring and other soft tissue injuries. The studies demonstrated that monitoring athletic training programs during the pre-season compared to the post-season utilizing wearable technology have assisted team athletic trainers and medical staff in developing programs to optimize player performance and minimize soft- tissue injuries.32

Impact detection The spongy nature of a woodpecker’s skull acts like a shock absorber by pinching the jugular vein to increase blood pressure in the brain to protect it from the 12,000 daily hammerings it performs on trees.58 Unfortunately, humans do not have any sort of ‘protection mechanism’ to mitigate or dissipate impact forces on the brain.58 The onset of concussions, brain injuries, and mental health illnesses caused by repeated trauma to the head have paved the way for newer technologies to detect and eliminate chronic traumatic encephalopathy (CTE).59 CTE is a neurodegenerative disease found in individuals who have experienced repeated traumatic brain injury (TBI) or concussions. In these conditions, stretching, compression, and shearing of axons during sudden brain movements over extended periods are hypothesized to cause axonal injury.60 The high incidence of such injuries in athletes is of major concern in modern collision sports.61

The American Academy of Neurology (AAN) defines a concussion as a “pathophysiologic disturbance in neurologic function characterized by clinical symptoms induced by biomechanical forces”.62 Guskiewicz et al. concluded that former NFL and collegiate football players who reported multiple concussions were at higher risk for depression and memory loss.63,64 Research on concussions and CTE is still rudimentary and primarily supported by clinical models.59 There remains a strong clinical need to develop devices, which could quantitatively and qualitatively measure impact forces on the brain to decrease the onset of concussions and reduce the incidence of CTE. Currently, work is being done to design custom personal protection equipment (PPE), such as helmets and mouthguards to improve player safety.65 Research by Stenger et al. and McCrory et al. showed the potential applicability of mouth guards towards preventing head and spinal injuries.66,67 Companies such as i1 Biometrics, Mamori, and Force Impact Technologies have devel- oped mouthguards that can monitor concussions (Table 2). The mouthguard by Force Impact Technologies contains embedded sensors, which relate collision intensity using color-coded LED’s (green, blue, or red) located at the front of the mouthguard. The colors are representative of the impact force delivered; green represents a mild impact, blue represents a medium impact force, and red represents a major impact force. The displayed color is then relayed via Bluetooth to the appropriate medical personnel in order to initiate the necessary protocols and interventions. The company believes the sensor placement will provide a high correlation back to the center of the brain. Despite the potential benefit of this technology, mouthguards are not universally used by all athletes. There remains a need to design and fabricate wearable sensors that can monitor and quantify impacts during collisions.68

D.R. Seshadri et al.

8

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

Several wearable device companies such as Noggin, Q30 Innovations, and X2 Biosystems have gained prominence in their ability to track, monitor, and prevent concussions. Noggin is focused on creating a protective skull cap whereby a gel cap generates friction with the inside of the helmet to hold it in place. This reduces slippage while dispersing and reducing impact forces on the head.69 The device also has a dry moisture wicking fabric that helps to protect athletes from heat-induced injury.69 Noggin has shown that its device can decrease impact forces up to 85% via a direct blow to the head when used with an approved helmet.69 Inspired by the woodpecker, Q30 innovations designed a device that prevents the brain from moving within the skull by clamping down on the jugular veins, causing the brain to swell and fit more snugly within the skull.70 Myers et al. tested the Q- collar device on youth hockey and high-school football players and successfully demonstrated that using the wearable device during live-game scenarios may have provided a protective barrier against the microstructural changes of the brain caused after repetitive head impacts.71,72 The studies used helmet acceler- ometers to track the number of hits a player received that had accelerations >20 g. Magnetic resonance imaging (MRI) was used to qualitatively observe and measure the diffusivity of water in different parts of the brain prior to and after the study. Although this device has not yet received FDA approval, it shows tremendous promise in reducing the incidence of concussions and TBI in collision athletes. The X-Patch Pro wearable sensor and X2 Mouthguard devices by X2 Biosystems are currently the most utilized head impact measuring devices in the sports commu- nity.73 The X-Patch Pro is an epidermal sensor containing an adhesive that can be worn behind the ear to record head impacts. The device transmits to a sensor data management (SDM) application on an electronic device.74 The sensor demonstrated a reduction in the incidence of head impacts leading to a decrease in concussions by 30–70% and is currently being utilized to study cumulative brain damage due to repeated head impacts.74 The sensors contain a tri-axial high-impact linear accelerometer and a triaxial gyroscope to capture six degrees of freedom for linear and rotational accelerations.73,74 X2 Biosystems utilizes proprietary analytical software called xSposure to relate acceleration measure- ments with impact duration, ranked from 1 (mild impact) to 10 (major impact).75,76 Additionally, the device calculates a Gadd Severity Index (GSI), head impact telemetry severity profile (HITsp), and generalized acceleration model for brain injury threshold (GAMBIT). Collegiate football teams at the University of Virginia75

and the University of Mississippi76 have utilized wearable devices by X2 Biosystems. Recently, professional football teams have adopted this device to monitor and track their own players.77

Research by the University of Virginia on their NCAA Division I-A football team compared the number and severity of sub- concussive head impacts sustained during helmet-only practices, shell practices, full-pad practices, and live-game scenarios to determine whether sub-concussive head impact on college athletes varies with practice type.75 The 20 participating football players wore the xPatch impact-sensing skin patches on the skin covering their mastoid process over the course of a season. Results showed that regulation of practice equipment could offer a viable and promising solution to drastically reduce sub- concussive head impact in collegiate football players. In another study, the University of Mississippi utilized the xPatch skin sensor to monitor head impacts on 15 NCAA Division I-A football players.76 After each practice, players reviewed their head impact profiles to determine the correlation between their head impacts relative to tackling technique and form. Results showed that the xSposure score of these players decreased by 15% over the course of the preseason.76 Wu et al. utilized high speed video to test a teeth-mounted mouthguard (developed by the research group in a previous study78), X-Patch Gen 2 soft tissue-mounted patch (adhered to the skin on the mastoid process), and the Reebok

elastic skull cap during sagittal soccer head impacts (Fig. 3d–f).79

The study focused on a 26-year-old male human subject who underwent soccer head impacts with clenched teeth. The ball traveled at an initial velocity of 7 m/s and was inflated to 8–9 psi.79

This velocity represented the average header speed in youth soccer. The researchers developed a method to quantify skull coupling of wearable head impact sensors in vivo. Furthermore, they found that in-plane skin patch acceleration peaked in the anterior–posterior direction and could be modeled by an under- damped viscoelastic system. The mouthguard showed tighter skull coupling than the other sensor mounting approaches (Fig. 3d–f). Additionally, the skin patch and skull cap had higher displacements from the skull compared to the mouthguard (Fig. 3d–f). Results from these studies demonstrated that wearable devices can track and minimize concussions; however, further clinical trials and a more in-depth understanding of the analytical platforms and modeling of sensor performance are needed to have a true clinical impact in sports. The work by Reebok is particularly interesting as it entails a partnership with MC10, a start-up originally out of the Rogers research group. The Reebok Checklight includes one or more accelerometers wired up with MC10’s “stretchable” electronics which consist of ultrathin gold electrodes that match the contour of the body.80 The partnership highlights the successful translation of scientific research into a commercial product to monitor impact forces on the head in a real-time manner.80 In another study, researchers developed a dry, textile-based nanosensor that detected early signs of TBI by continuously monitoring various neural behaviors indicative of the injury, such as drowsiness, dizziness, fatigue, sensitivity to light, and anxiety.81 The device comprised of a network of flexible sensors woven or printed into a skullcap worn underneath a football helmet. The device used Zigbee/Bluetooth wireless telemetry to relay the data from the sensors to a receiver and to a remote monitor. The system included a pressure-sensitive textile sensor embedded underneath the helmet's outer shell, which measured the intensity, direction, and location of the impact force. The other sensors worked as an integrated network within the skullcap and included a printable and flexible gyroscope that measured rotational motion of the head and body balance and a printable and a flexible 3-D accelerometer that measured lateral head motion and body balance. Additionally, the device was outfitted with physiological sensors to detect pulse rate and blood oxygen levels. At time of publishing, the device had yet to be tested in real-time football games. While follow-up clinical data is not available, assessing the use-case of such devices in randomized, controlled studies is necessary to further translate such technology to improve athletic safety and performance.

Biomechanics detection Motion analysis to study biomechanics is currently performed by measuring body kinematics via motion capture systems such as optical, inertial or magnetic units (IMU), electrogoniometers, and mechanical tracking.82 However, their disadvantages prevent them from being utilized for an extended duration to monitor human movement. Optical systems are expensive, require complex setups, and data processing systems, and are rarely used to assess a single joint or body part. IMU-based systems have limited fields of operation, high error rates, and high sensitivities. Mechanical tracking systems have poor portability and cannot be used during real competition situations. Therefore, there is a strong need to develop alternative technologies to monitor and quantify human-body kinematics in a non-invasive and accurate manner. Epidermal wearable sensors can play a key role in quantifying

human movement and monitoring changes in joint mechanics in order to prevent injuries. Key properties of these sensors for biomechanical detection include their high stretchability,

D.R. Seshadri et al.

9

Scripps Research Translational Institute npj Digital Medicine (2019) 71

flexibility, robustness, and durability.9,83,84 These sensors can be applied over the joint as a sleeve to monitor the stress and strain on the elbow, anterior cruciate ligament (ACL), medial collateral ligament (MCL), or posterior cruciate ligament (PCL). In 2015, 25% of active major league baseball (MLB) pitchers and 15% of minor league pitchers underwent ulnar collateral ligament (UCL) reconstruction. Often referred to as Tommy John surgery,85 the UCL in the medial elbow is replaced with either an autograft or donor allograft tendon. This reconstructive surgery typically sidelines a pitcher for the entire season due to the time- intensive rehabilitation that follows.86 Data from Motus Global in 2015 showed that pitchers represented ~59% of injured players in MLB and $420 million in sidelined salaries.85 The “motusBaseball Sensor,” developed by Motus Global, is the first wearable device approved by the MLB for in-game use.87 The wearable sleeve quantifies the strain exerted by a pitcher to gain a better understanding of the factors that cause ligament damage.88 The device contains five sensors and an analytical program to view the biomechanical data. A single sensor near the elbow measures the stress exerted on the UCL. The sensor also has clinical utility for football quarterbacks (QBs). Football quarterbacks exhibit overarm throwing injuries due to overuse and require rehabilitation therapy from injuries on the throwing arm caused by contact. A recent paper by Motus Global tested the motusBaseball Sensor on a high school male baseball pitcher.89 The athlete was instru- mented with 46 reflective markers on anatomical locations and kinematic data were collected at 480 Hz using a 12-camera 3D motion capture system (Motion Analysis Corp, mocap). The motusBASEBALL sensor was placed on the inside of the forearm ~3 cm distal to the medial elbow epicondyle.89 The athlete pitched a baseball off a mound into a net at a distance of ~5m away from the pitching rubber. Following this, the athlete made seven throws with a football in a “shotgun” stance (e.g. no drop- back prior to throwing). Full body kinematics were used to calculate elbow valgus torque by both mocap and the motusBA- SEBALL sensor. The sensor read slightly higher peak elbow valgus torque for baseball pitching (3%) and slightly lower in football throwing (5%) (Fig. 3g).89 The results demonstrated that the sensor was successful in calculating maximum elbow valgus torque in both baseball and football throwing scenarios (Fig. 3g).89

While statistical analysis was not performed, the authors of the study concluded that the differences between the mocap calculations of torque and sensor calculations of torque were minor. The study showed that the motusBASEBALL sensor provided an accurate measure of elbow valgus torque for both baseball pitching and football throwing (Fig. 3h).89 Use of this data from the sensor could enable measures of acute and chronic workloads that are joint specific to the throwing arm to improve performance and minimize injury. We hypothesize that such information could enable coaches to refine throwing techniques to serve as “coachable moments” for athletes specializing in throwing-based sports to minimize serious long-term injury. Detecting biomechanical forces and arm angles have been utilized as teaching tools in sports as well. Vibrado Technologies has developed a wearable sleeve that measures arm angles and movement to model shooting motion in basketball90 (Table 3). This device has potential applications in basketball training and other sports where “muscle memory” is crucial for repeated success. The ACL is the primary stabilizing knee ligament that prevents

anterior translation of the tibia.91 ACL tears are one of the most common knee injuries observed in sports medicine. Forces in the ACL can be studied and quantified in six degree of freedoms (DOF) due to externally applied loads.91 An accurate device to measure the biomechanics to determine the correlation between the ACL and the kinematics of the knee is necessary for the longevity of athletes and for sports trainers and physicians to better tailor rehabilitative therapy for the athlete.91 Currently there

is no quantifiable method or commercial device to determine ACL strain. Thus, the development of a robust sensor capable of such measurements is highly desirable for cutting and pivoting sports, such as soccer, football, basketball, and rugby.3 In a recent study, a wearable inertial-based device to evaluate ACL injury risk during jumping tasks was designed.92 The accuracy of the sensor was measured by comparing temporal events (initial contact, toe-off), jump height, and sagittal plane angles (knee, trunk) to simulta- neous measurements obtained with a marker-based optoelec- tronic reference system on 38 healthy subjects. Overall, the wearable system demonstrated good concurrent validity with marker-based measurements and good performance in terms of the known risk factors for ACL injury. However, the obtrusive nature of the device severely hindered its utility for use during team-based activities thus necessitating significant modifications (e.g. miniaturized and unobtrusive form factor) for the athlete.

PHYSIOLOGICAL STATUS OF THE ATHLETE TO OPTIMIZE ON- FIELD PERFORMANCE Heart rate and electrocardiogram detection Heart rate (HR) is a key indicator of physiological adaptation, exercise intensity, and workout effort.93 Standard HR monitors are comprised of a transducer worn around the chest that can store data locally for 1–2 weeks or alternatively transmit the data to a wireless wrist display.94 Newer detection methods, such as photoplethysmography (PPG), utilize optical sensors to detect HR directly from the wrist or fingertip by detecting blood volume changes as a function of transmitted and reflected light. Prior studies showed that analyzing HR data allowed researchers to more accurately quantify physical activity in individuals. On that note, researchers examined the relationship between HR and maximal oxygen consumption (VO2) during field and laboratory- based moderate intensity workouts.93 Energy expenditure (EE) was estimated from HR data by adjusting age and fitness by expressing the EE data as a percentage of HR reserve (HRR) and a percentage of VO2 reserve (VO2R).

93 Results demonstrated that HR was a relatively accurate predictor of EE (r= 0.87). Current HR devices on the market include the Xiaomi mi Band,

Apple Watch, Garmin and Fitbit devices, Komodo AIO Smart Sleeve, Jabra Sports Pulse headphone, WHOOP Band, and the Zephyr Bioharness™ (Table 4). The Komodo AIO Smart Sleeve contains a processor and internal memory incorporated within the fabric to collect information about an individual’s HR, sleep quality, and workout intensity. The sleeve contains a conductive liquid that connects one electrode on the user’s wrist to another electrode under the arm.95 Clinical validation of this device is needed to assess its efficacy for athletes. The Jabra Sports Pulse headphones have ushered in a new wave of wearable devices referred to as “hearables”. Jabra’s HR sensor lies against the bottom of the inside rim of the left ear canal. Studies from Jabra have shown that HR readings are 99.2% comparable to ECGs.96

Additionally, wearable sensors have been designed for the concurrent detection of various physiological parameters. The Zephyr Bioharness™ can simultaneously measure five physiologi- cal and activity-related parameters, such as HR, breathing frequency, skin temperature, tri-axial movement, and posture in real-time.97,98 The sensor is affixed to clothing via a strap and worn around the abdomen. The Bioharness™ is being utilized for physical activity and exercise monitoring, emergency situations, and for monitoring the well-being of military personnel.58,99

Despite demonstrated use of wearable sensors for HR detection, the assessment of their accuracy (defined as the statistical difference between actual and reported data) is still limited. A study by Wang et al. highlighted these limitations in a number of wrist-worn heart monitors, such as Polar H7, Apple Watch 3, Mio Fuse, Fitbit Charge HR, and Basis Peak.100 Heart rate readings were

D.R. Seshadri et al.

10

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

compared to a gold-standard ECG and it was found that the Polar H7 device had the highest concordance correlation coefficient of 0.99 followed by the Apple Watch (0.91), Mio Fuse (0.91), Fitbit Charge HR (0.84), and Basis Peak (0.83). The study found that none of the wrist-worn devices achieved the accuracy of a chest-strap- based monitor. Additionally, these devices were most accurate when measuring resting HR and became less accurate with increasing exercise. Gillinov et al. assessed the accuracy of four optically based HR wrist-monitors (Apple Watch, Fitbit Blaze, Garmin Forerunner 235, and TomTom Spark Cardio), one on each wrist, and one forearm monitor (Scosche Rhythm+) compared to an ECG chest strap monitor (Polar H7) during various types of aerobic exercise.101 Fifty healthy adult volunteers performed exercise protocols on a treadmill, a stationary bicycle, and an elliptical trainer (arm movement). HR was recorded at rest, light, moderate, and vigorous intensity for each exercise. Agreement between the HR measurements from the wearable sensor and that of ECG was assessed using Lin’s concordance correlation coefficient (rc). The chest strap monitor (Polar H7) had the best agreement with ECG (rc= 0.996) across all exercises followed by the Apple Watch (rc= 0.92), the TomTom Spark (rc= 0.83), and the Garmin Forerunner (rc= 0.81), Scosche Rhythm+ (rc= 0.75), and Fitbit Blaze (rc= 0.67). All devices performed well (rc= 0.88–0.93) on the treadmill except the Fitbit Blaze (rc= 0.76). During cycling, only the Garmin, Apple Watch, and Scosche Rhythm+ had acceptable agreement (rc > 0.80). On the elliptical trainer without arm levers, only the Apple Watch was accurate (rc= 0.94). None of the devices were accurate during elliptical trainer use with arm levers (all rc < 0.80). The study found that the accuracy of wearable,

optically based HR monitors varied with exercise type and was greatest on the treadmill and lowest on the elliptical trainer. The team concluded that electrode-containing chest monitors should be used only when accurate HR measurement is needed. In another similar study, Stahl et al. evaluated the accuracy of the Scosche Rhythm (SR), Mio Alpha (MA), Fitbit Charge HR (FH), Basis Peak (BP), Microsoft Band (MB), and TomTom Runner Cardio compared to the Polar RS400c HR chest strap among 50 healthy volunteers (Fig. 4a).102 The study protocol entailed having the volunteers on a treadmill for 30 min walking at various velocities for 5 min each (3.2, 4.8, 6.4, 8.0, and 9.6 km/h). Interestingly, the study by Stahl et al. showed that wearable activity trackers provided an accurate measure of HR during non-stationary activities. The mean absolute percentage error values were: 3.3%, 3.6%, 4.0%, 4.6%, 4.8%, and 6.2% for the TT, BP, RH, MA, MB, and FH wearable wrist-devices, respectively. The Pearson product–moment correlation coefficient (r) was calculated: r= 0.959 (TT), r= 0.956 (MB), r= 0.954 (BP), r= 0.933 (FH), r= 0.930 (RH), and r= 0.929 (MA). Results from a 95% equivalency test showed that monitors were found to be equivalent to those of the criterion HR (±10% equivalence zone: 98.15–119.96). The authors of this review hypothesize that these deviations can be attributed to various factors, such as the modality of the PPG signal compared to that produced from an ECG (a key difference between wrist-monitors versus epidermal patches), the difficulties associated with peripheral wrist location, and the noisy interface of the skin.103,104 Further clinical validation and standardization of clinical protocols may be necessary to homogenize results among

Fig. 4 Wearable sensors monitor the physiological status (heart rate, muscle oxygen saturation, and sleep) of the athlete. a Bland Altman plots for all wearable wrist-sensors compared to the Polar RS400. x-axis: Mean of PolarRS400 and tested device; y-axis: PolarRS40 and tested device. b SmO2 results for a representative subject during an incremental cycling test. The Humon Beta SmO2 (red line) and MetaOx SmO2 (green line) absolute values are 3–5% different; however, the overall trend holds for the duration of the exercise. The vertical lines indicate the time point that the power on the bike was changed and the numbers on top of the graph represent the power level (Watts) at which the subject was cycling. c Mean absolute percent error for various wearable devices during total sleep time. The numbers denoted next to each bar represent the mean absolute percentage error values which were used to calculate the absolute difference between each monitor and the sleep diary values. Figures were reproduced with permission from Stahl et al.102 a, Farzam et al.117 b, and Lee et al.131 c

D.R. Seshadri et al.

11

Scripps Research Translational Institute npj Digital Medicine (2019) 71

clinical trials to accurately improve patient outcomes via the use of such technology. There has been a growing interest to develop epidermal

electronics to monitor HR, leveraging advancements in flexible materials and health analytics to mitigate accuracy-related issues posed by wrist-worn devices. Epidermal patches such as the BioStamp MD™, Healthpatch MD, Vital Scout, and Zio XT Patch, have emerged as promising options for monitoring the HR of athletes. Kabir et al. utilized the MC10 Biostamp to develop an optimal configuration of the sensors, which would provide the best agreement with the Frank orthogonal ECGs for long-term monitoring of a vectorcardiogram (VCG).105 A VCG represents the movement of the heart vector in three orthogonal dimensions and provides information complementary to that of a 12-lead ECG.106

Analysis of VCGs has been demonstrated to help define abnormal electrophysiological substrate in patients with life-threatening ventricular arrhythmias and sudden cardiac death (SCD).106 The study by Kabir et al. evaluated parameters, such as the QRS-T angle, spatial QRS, and T-vector characteristics, and other global electrical heterogeneity parameters in 50 subjects.105 Each subject underwent 10 s of orthogonal ECG followed by 3–5min of ECGs using the Biostamp patches and MAC 5500 ECG system at five locations on the torso while at rest in a sitting position. Results confirmed that the biostamp patches could be used for the long- term monitoring of the VCG parameters previously described. Utilizing devices like the Biostamp demonstrates the utility of epidermal ECG patches to monitor those at risk of arrhythmias in a non-intrusive and continuous manner. Translation of such technology to monitor athletes suffering from cardiac conditions, such as hypertrophic cardiomyopathy or atrial fibrillation present next steps to further enhance the value of wearable sensors for sports. The American College of Cardiology (ACC) recently established the Sports and Exercise Council.107 An important objective of this council is to define the essential skills necessary to practice effective sports cardiology to track competitive athletes and highly active people (CAHAP) who may be most at risk for adverse cardiovascular outcomes during intense physical activity.107

Building upon the work by Kabir et al. clinical adaptation of devices like that of the Vital Scout patch by VivaLNK in sports such as rowing could diversify the use case of such technology to proactively monitor the health of athletes in a real-time manner to gain insight regarding their heart rate, respiration rate, stress levels (as a function of heart rate variability, HRV, as discussed later in this review), sleep, and activity.108 In a recent study, Lee et al. fabricated a self-adhesive ECG patch that conformally laminated onto the wrinkles of the skin, maintained robust contact, and self- adhered onto the epidermis.109 The epidermal device recorded various biosignals from an ECG, EMG, and electrooculogram (EOG) while avoiding skin irritation. The team developed a multi-material dry adhesive utilizing polydimethylsiloxane (PDMS) and carbon nanotubes (CNTs), leveraging the biocompatibility and excellent mechanical properties of the polymer and electrical conductivity of the CNTs. The device showed promise for monitoring ECG signals long-term. In another study, researchers developed a multifunctional epidermal device capable of measuring biosignals from both ECG, EMG, and temperature.110 The sensor contacted the skin directly via an elastomeric stamp which was transfer printed onto the skin via the application of an acrylate/silicone spray-on-bandage. The study demonstrated that the application of advanced materials coupled with integration methodologies resulted in a viable multimodal epidermal sensor for monitoring responses through and on the skin. In another study, Hu et al. developed a conductive elastomeric electrode devoid of con- ductive pastes for the measurement of an ECG signal.111 PDMS was mixed with other biocompatible conductive nanoparticles to improve the electrical conductivity of the substrate. A micro- replica mold casting for the micro-structures was applied to

reduce the micro-structural deformations along the direction of signal transmission to maintain the corresponding electrical impedance under the physical stretch by the movement of the human body. The gel-less electrodes provided a more convenient and stable bio-potential measurement platform when tested on a healthy human subject undergoing walking and running tests. Further work translating such technologies on athletes during real-time practices remains the next step before the efficacy of such devices is deemed ready for everyday use. Specifically, there remains a need to evaluate the build-up of eccrine sweat underneath the substrate of such devices to study skin health and sensor adhesion over a prolonged workout. Another emerging area to monitor HR for athletes is the

incorporation of sensors into textiles to form “smart garments”. Researchers fabricated a wearable ECG monitoring garment that utilized electrodes from carbon-derived paste.112 The paste was applied to the skin and dried for 5 min resulting in a flexible and detachable electrode. The electrodes were connected to an ECG affixed to the garment and used to measure ECG signals during walking and running. Despite the promising research, we assessed that issues such as reliability of electrode connections (as a function of player movement), adhesive robustness and longevity (due to eccrine sweat generated during workouts), and the effects of impact on sensor reliability need to be considered during preliminary studies to test if the device is capable of monitoring real-time performance in sports. On the commercial front, Kymira, a smart textile company, launched an early prototype, currently in final development, of its cardiac monitoring t-shirt designed to lower the risk of heart attacks in athletes.113 The shirt wirelessly transmits the athlete’s heart rhythm to a mobile phone via Bluetooth where it can identify an unusual heart rhythm that could lead to sudden cardiac arrest. Printed electrodes on the shirt’s fabric feed into a processing unit which transmits and rectifies the ECG data. The textiles in the shirt regulate body temperature to improve athletic performance. Furthermore, minerals embedded in the fabric capture energy produced by the body during exercise and re-emit that back as infrared (IR) energy into muscle. This has been shown to increase circulation, increase tissue oxygen levels up to 20%, and provide pain relief to reduce muscle soreness. The value in HR training is in the use of zones, which are all

based off an HR value that is relative to the maximum HR. Despite its potential value for sports, there remains multiple issues with HR monitoring.114 Firstly, maximum HR is often calculated using the formula “220 minus the age of the person”, or a slight variation to that. The physical fitness, body composition, or other individual variances which could affect this maximum HR value are not considered. Secondly, HR is dependent on multiple external factors, such as caffeine intake or sleep. Thus, HR may be unreliable unless it is measured under extremely controlled environments. Lastly, HR is a systemic measurement. It provides information on how the heart is adjusting to activity; however, it fails to provide specific information about how the working part of the body is responding to exertion. As such, measurement of an HR value currently provides little to no use when athletic trainers want to quantify the workout of the athlete to determine the ACWR and to predict the incidence of soft-tissue injuries during the rigors of training camp or live performance. As reviewed next, measurement of muscle oxygen saturation (SmO2) levels has been shown to circumvent such hurdles.

Muscle oxygen saturation Physiological quantification of how muscles respond to physical exercise is gaining interest in elite-level athletes to improve their overall performance. In the past, athletes have relied upon measurements, such as blood lactate concentration, HR, or maximum oxygen uptake (VO2max) to assess the intensity at

D.R. Seshadri et al.

12

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

which they should be exerting themselves.115,116 While quantifica- tion of these parameters has helped craft athlete-specific workout regimens to improve performance, these measurements are indicative of systemic changes in the body, with no detailed information about the working muscle groups.117

Muscle oxygen saturation, which refers to the amount of oxygen in the blood of muscles, is a measurement that has emerged as a useful parameter to help athletes optimize their performance. The technology behind SmO2 monitors was devel- oped several decades ago; however, it remains an emerging area for wearable device fabrication in scientific literature today. Commercial wearable, fiberless devices include the Humon Hex, Moxy Monitor (Fortiori), and Portamon (Artinis Medical System).117

These devices work by non-invasively measuring the amount of oxygenated and deoxygenated blood in the muscles using light waves.117 The Moxy Monitor and Portamon devices can be manually strapped on to any muscle group and have been used during a variety of activities including cycling, running, and strength training.117 The Humon Hex is 6.0 × 5.7 × 1.4 cm in size and is placed over the quadriceps using a Velcro strap that hooks through the device117 (Table 4). The device communicates with a smartphone via Bluetooth and a custom app that displays the workout progress in real-time. Muscle oxygen monitors (such as the ones mentioned) use optical techniques to measure the oxygenated hemoglobin concentration (HbO2) and deoxygenated hemoglobin concentration (Hb) in the muscle during exercise.117

These devices are able to do this by shining near-infrared light (NIR, 0.7–1.4 µm wavelength) into the muscle and by detecting reflected light. Hemoglobin concentrations can be quantified by measuring the amount of light that is absorbed.118,119 The parameter that is typically reported to the athlete is called muscle oxygen saturation (SmO2), which is the ratio of HbO2 to total hemoglobin concentration (HbT), where HbT is the sum of HbO2

and Hb.114,117 As the muscles exert more energy (work harder), more oxygen is used and the SmO2 level decreases.120,121

Therefore, SmO2 provides athletes with a localized measurement for how muscles are performing during an activity. Some of the benefits from using SmO2 are in (1) measuring localized muscle performance, (2) determining whether the working muscles are being exerted beyond their limits to inform the athlete that their muscles are running low on oxygen and they (athlete) cannot sustain the current activity, and (3) evaluating muscle recovery. SmO2 can show the rate at which the oxygen is delivered back into the muscles and when the muscles are ready to perform again.122 There are two primary factors that influence SmO2

measurement throughout exercise: oxygen delivery and oxygen consumption.120,121 As the muscles exert more effort, they demand more oxygen thus increasing blood supply. This increase is accomplished primarily by an increase in HR. However, there is a level when this increase in blood supply can no longer match the oxygen demand within the muscle, which results in a decrease in SmO2 levels. When the athlete slows down during the recovery phase of an interval set, the SmO2 increases due to the lower oxygen requirement in the muscles in addition to the high blood supply still being present.117

A key question left unanswered in the field is: Can the integration and translation of SmO2 levels measured from wearable sensors into internal workload models (HR or sRPE) better monitor athletic performance and predict soft-tissue injury in elite-level athletes? As previously mentioned, the sRPE is based on the intensity of the session on a 1–10 ‘rating’. However, it does not consider key physiological parameters which play a crucial role in the performance of the athlete. We hypothesize that the stratification of physiological parameters (e.g. SmO2 levels) into a scale analogous to that of current sRPE ratings to calculate player loads would enable intrinsic workload measurements to be based on physiological parameters which affect workout intensity and performance rather than a scale lacking formal clinical guidelines

and variability among athletes. We predict that development of such models from sensor data could serve as the next-gold standard to accurately and efficiently assess human performance. Clinical validation of current devices against benchtop technol-

ogy is needed to enable this translation. Farzam et al. compared the SmO2 levels by the Humon Hex (beta device) against a benchtop fiber-based frequency-domain NIR (FDNIRS) system (MedaOx, ISS) on the rectus femoris muscle among 14 male and 3 female athletic subjects on a cycle ergometer.117 The goal of the study was to examine the accuracy of the Humon Hex to understand potential limitations in measuring SmO2 levels. The authors studied the real-time feedback from the Humon device and reported variations between the optically derived threshold and blood lactate threshold during an incremental cycling test. The rectus femoris was selected to maximize fiber movement, since this was the area of the leg where the fibers remained the most stable. In addition to the body mass index (BMI) calculated for each subject, the subcutaneous adipose tissue thickness (SCATT) on the rectus femoris of the right quadricep was measured using a skinfold caliper before the start of the cycling test. Blood lactate levels were calculated based on a combination of HbO, HbR, HbT, and SmO2 levels and were displayed among corresponding exercise zones (green, orange, red, and blue). The green zone indicated a steady state. The orange zone indicated the athlete is approaching their limit. The red zone indicated the athlete has hit or exceeded their limit. The blue zone indicated that the athlete is in recovery phase. Overall, the study validated the performance of the Humon Hex wearable device against the MedaOx benchtop system (Fig. 4b). The wearable device provided similar results to more expensive FDNIRS technology. The main limitations to all continuous wave (CW) and FDNIRS systems is the reduced sensitivity to muscles in the presence of subcutaneous adipose tissue.117 Focusing on athletes who tend to have thin adipose layers provides a larger drop in SmO2 levels than what can be achieved of those with thicker adipose layers.117

Monitoring the sleep quality of the athlete is instrumental for the athlete to maintain a healthy HR and SmO2 range necessary to maximize performance.

Sleep quality detection Sleep quality and duration is an important measure of health and is known to directly affect the performance and recovery of an athlete. Wearable devices have been developed to evaluate sleep quality and have focused on monitoring body movement patterns as a measure of sleep restfulness. Examples of wearable devices currently in the market which monitor and track sleep quality are the Fitbit sensors, Jawbone UP, Misfit Shine, Komodo AIO Smart Sleeve, Polar watches, and WHOOP band (Table 5). The WHOOP band is the first wrist-based device to proactively prescribe to the user the hours of sleep needed to ensure a full recovery. The device measures physiological markers (e.g. resting HR, HRV) to indicate strain, optimize recovery, and maximize performance on a daily basis.123 Based on this data, the algorithmic platform determines the physical exertion during workouts over the course of a day and utilizes this data to estimate the number of hours of sleep required for a full recovery.123 A recent study, funded by WHOOP, utilizing the WHOOP band, compared changes in and relationships between resting HR, HRV, and sleep characteristics in 10 NCAA Division 1 collegiate female cross-country athletes over a 12-week season.124 Resting HR at the end of the season showed a meaningful increase compared to the beginning of the season. Higher resting HR and lower HRV were associated with an increase in percentage of time spent in a slow wave sleep. The data suggested that when the physiological restorative demand was higher, the percentage of time in slow wave sleep was increased to ensure recovery. The study demonstrated that monitoring sleep using devices like the WHOOP band enabled the implementation

D.R. Seshadri et al.

13

Scripps Research Translational Institute npj Digital Medicine (2019) 71

of sleep hygiene strategies to promote adequate slow wave sleep when the body needed physiological restoration. Randomized controlled studies comparing the WHOOP band to other devices utilizing larger sample sizes among athletes is greatly needed to validate the efficacy of such technology for athletes. Various studies have shown that a lack of sleep lowered athletic

performance, worsened lung function, decreased the time to fatigue, increased injury risk, and increased lactic acid production thereby increasing the likelihood of post-workout muscle fatigue and soreness.125–130 A recently published study assessed the accuracy of commercially available sensors in 78 adults with a mean age of 27.6 ± 11 years by estimating sleep measurement with a sleep diary (SD) for three nights.131 Results showed that the greatest equivalence with the SD for total sleep time were the Jawbone UP3 and fitbit charge heart rate with effect sizes of 0.09 and 0.23, respectively (Fig. 4c).131 Other tested wearables such as SenseWear Armband, Garmin Vivosmart, and Jawbone UP3 produced the greatest effect sizes of 0.09, 0.16, and 0.07 respectively.131 Rosenberger et al. assessed the accuracy of nine wearable devices (Actigraph GT3X+, activPAL, Fitbit One, GEN- Eactiv, Jawbone Up, LUMOback, Nike Fuelband, Omron ped- ometer, and Z-Machine) over a 24-h period in their ability to accurately track sleep.132 The sedentary behavior (SED), light intensity physical activity (LPA), and moderate-to-vigorous physi- cal activity (MVPA) were measured. The Z-Machine utilized three electrodes applied to the head/neck to measure sleep. The other devices were worn on the wrist and relied on an accelerometer- based measurement algorithm to estimate total sleep time. LUMOback and activPAL did not have specific sleep measurement because sedentary time and sleep were recorded based on posture and were excluded from sleep measurements. The Fitbit device was moved from the trunk to the wrist and placed over the forehead for sleep measurement. The subject then pressed and held a button on the device to enable sleep mode. Similar procedures were used for the Jawbone, GT3X+, and GENEactiv devices. Comparisons (to standards) were derived for total sleep time (Z-machine), time spent in SED (activPAL), LPA (GT3x+), MVPA (GT3x+), and steps (Omron). Error rates ranged from 8.1–16.9% for sleep, 9.5–65.8% for SED, 19.7–28.0% for LPA, 51.8–92% for MVPA, and 14.1–29.9% for steps. The GT3X+ device had the closest measurement for sleep, LUMOback for sedentary behavior, GENEactiv for LPA, Fitbit for MVPA and GT3X+ for steps. The study concluded that no device accurately captured activity data across an entire day. Polysomnography (PSG) remains the gold standard for monitoring sleep and should be utilized in clinical studies to quantify the efficacy of a wearable device to track sleep.133 PSG involves recording multiple physiologic variables, including EEG, ECG, EMG, and electro-oculogram (EOG), which is then scored by human examiners based on standardized criteria.134 While PSG recordings provide an accurate measurement of sleep quality, their high cost make it impractical to implement within a long-term sleep monitoring system. Furthermore, attaching numerous sensors to an individual’s body is considered intrusive, and may in turn disturb sleep. It has been hypothesized that wearable sensors could bridge this gap. Mantua et al. assessed the reliability of wrist-worn wearable devices, such as the Basis Health Tracker, Misfit Shine, Fitbit Flex, Withings Pulse O2, and a research-based actigraph, Actiwatch Spectrum against PSG.135 A Wilcoxon Signed Rank test was used to assess differences between devices relative to PSG and to correlate the strength of the data. Data loss was greatest for the Fitbit and Misfit devices. For all the devices, the authors found a strong correlation of total sleep time with PSG; however, sleep efficiency differed from PSG for the Withings, Misfit, Fitbit, and Basis devices. Data from the Actiwatch did not differ from that of PSG. A weak correlation in sleep efficiency (time asleep/time in bed) was noted from Actiwatch correlated with PSG. Light sleep time differed from PSG (nREM1+ nREM2) for all devices. Measures of deep sleep time

did not differ from PSG (SWS+ REM) for the Basis device. While total sleep time, and in some cases, sleep efficiency, can be monitored via wrist-worn, devices, the reliability of these sensors remains low. Furthermore, the authors concluded that these devices did not yet yield sufficient information for accurate sleep staging, even on a superficial level (e.g., light vs. deep). The authors concluded that PSG should remain the mainstay when monitoring the sleep of individuals. Further studies of devices against PSG are necessary to test the clinical relevancy of this technology for elite-level athletes.

NEXT STEPS: WEARABLE SENSORS ASSIST IN THE RETURN TO PLAY FOR ATHLETES Sports medical personnel are faced with return-to-play (RTP) decisions for every athlete who want to return to activity at the highest level.136 The myriad of factors related to history, physical examination, testing, workload and intensity, and baseline characteristics of the athlete can make the RTP decision-making process complex and challenging.136 The RTP decision-making process was authored in a three-step protocol referred to as the Strategic Assessment of Risk and Risk Tolerance (StARRT).136,137

Step one outlines the medical factors associated with the injury to determine the level of injury severity with the RTP.136 Step two focuses on the player or sport factors that may mitigate or augment the risk of injury or reinjury.136 Step three is focused on the factors associated with whether the final ascertained risk is worth taking within the confines of the needs of the coach, team, athlete and medical service provider.136 While inclusion of steps one and three are key to the final decision, our discussion in this review was focused on step two, specifically on the initial evaluation and monitoring of the athlete to assess their performance and risk of initial or reoccurring injury. Our discussion in this review was centered around the ability

of the musculoskeletal, cardiopulmonary, and psychological systems of the athlete to be assessed and quantified in a non- invasive and unobtrusive manner to restore sports-specific skills and function. We believe a key aspect that has been excluded from the RTP decision is related to quantitatively measuring the amount of training the athlete has completed over the time of the recuperation or during high-acuity training periods to ultimately enable the athlete to be mentally and physically prepared for the physical and mental demands of the game (Table 6). A central theme which permeated throughout this review was that the use of wearable sensors can enable medical personnel and athletic trainers to monitor the biomechanical and physiological status of the athlete to mitigate or minimize the onset of injuries and assess athlete performance in a real-time manner. Risk of athlete participation is dependent on the interaction between tissue health (biomechanical, physiological, or mental stress the tissue can absorb) and tissue stresses (biomechanical, physiological).136 This risk is then compared with the clinician’s and/or athlete’s risk tolerance, which is a function of numerous factors related to the overall health of the athlete.136 After all factors considered, if the risk assessment is less than the risk tolerance, the decision should be to RTP.136

Conversely, if the risk assessment is greater than the risk tolerance, the decision should not be to RTP.136

In summary, this paper discussed the utility of wearable sensors to measure biomechanical and physiological parameters affecting athlete performance. Specifically, the first section on physical performance and safety included sensors which measure position and motion, impact, and biomechanical forces. The second section pertaining to the physiological status of the athlete included sensors which measure heart rate, muscle oxygen saturation, and sleep quality. In each section, we provided examples to discuss how such technology has been utilized or could be adapted by

D.R. Seshadri et al.

14

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

Ta bl e 6.

Su m m ar y o f m et h o d s u ti liz ed

o r em

er g in g to

q u an

ti fy

at h le te

tr ai n in g lo ad

to m o n it o r re co

ve ry

an d p er fo rm

an ce

M et h o d

U se d to d ay

in sp o rt s

W ea ra b le s u ti liz ed

M et ri cs

A d va n ta g es /d is ad

va n ta g es

Q u es ti o n n ai re

Ye s

N o

Ve rb al

o r w ri tt en

fo rm

A dv an

ta ge :E

as y to

co n d u ct

D is ad

va nt ag

e: H ig h ly

va ri ab

le ;o

ft en

in ac cu

ra te

Se ss io n -r at e o f

p er ce iv ed

ex er ti o n

Ye s

N o

Sc al e fr o m

1 to

9 d et ai lin

g in te n si ty

o f

w o rk o u t. Sc al e u se d in

co n ju n ct io n w it h

w o rk o u t d u ra ti o n to

d et er m in e lo ad

A dv an

ta ge :E

as y to

as se ss

D is ad

va nt ag

e: H ig h ly

va ri ab

le ;o

ft en

in ac cu

ra te

B lo o d la ct at e

Ye s (e m er g in g )

N o

C o n ce n tr at io n

A dv an

ta ge :U

se d to

p re d ic t an

ae ro b ic th re sh o ld

(k ic ks

in w h en

ex er ci se

is in cr ea se d an

d th e ae ro b ic

sy st em

ca n n o lo n g er

ke ep

u p w it h th e b o d y’ s en

er g y sy st em

D is ad

va nt ag

e: C o st ,i n ef fi ci en

t, ti m e- va ry in g p ro ce ss

Tr i-a

xi al

ac ce le ro m et er s

an d G PS

Ye s

Ye s: C at ap

u lt ,Z

eb ra

A cc el er at io n ,l o ca ti o n ,a

n d ve

lo ci ty

u se d to

co m p u te

Pl ay er Lo

ad (a rb it ra ry

u n it ) to

d er iv e A C W R

A dv an

ta ge :E

as y to

u ti liz e

D is ad

va nt ag

e: Va

ri ab

ili ty

in se n so r te ch

n o lo g y co

u ld

le ad

to in ac cu

ra cy .N

ee d to

d ev el o p al g o ri th m s to

fi lt er

n o is e (e .g .p

la ye r m o vi n g o n th e si d el in e

co m p ar ed

to o n -fi el d p er fo rm

an ce )

H ea rt

ra te

N o

Ye s: A p p le

W at ch

,F it b it ,P

o la r

Ti m e in

H R zo n es ,H

R V

A dv an

ta ge :E

as y to

co lle ct

la rg e d at a se ts

fo r ro b u st

an al ys is

D is ad

va nt ag

e: Va

ri ab

ili ty

in se n so r te ch

n o lo g y co

u ld

le ad

to in ac cu

ra cy .S

en so r lo ca ti o n at tr ib u te d to

d ev

ia ti o n s.

M u sc le

o xy g en

sa tu ra ti o n

N o

Ye s: H u m o n H ex

Sm O 2 le ve ls st ra ti fi ed

in to

w o rk o u t zo n es

A dv an

ta ge :E

as y to

co lle ct

la rg e d at a se ts

D is ad

va nt ag

e: N ee

d fo r va lid

at io n o f m o d el s

B io ch

em ic al

co n ce n tr at io n 5 – 8

N o

N o d ev

ic es

u se d to

m o n it o r tr ai n in g lo ad

an d

re co

ve ry

d ir ec tl y. In d ir ec t m ea su re s in cl u d e

m o n it o ri n g h yd

ra ti o n le ve ls an

d sw

ea t ra te

C o n ce n tr at io n

A dv an

ta ge :I n si g h t in to

th e b io ch

em is tr y o f th e

at h le te

to p re d ic t h yp

o h yd

ra ti o n ,h

yp o n at re m ia ,a

n d

fa ti g u e.

D is ad

va nt ag

e: Te ch

n o lo g y st ill

d ev

el o p in g .N

ee d to

d ev el o p p re d ic ti ve

an al yt ic s b as ed

o n th e

b io ch

em ic al

p ro fi le

o f th e at h le te

D.R. Seshadri et al.

15

Scripps Research Translational Institute npj Digital Medicine (2019) 71

the sports community to enable athletes to perform better, recover faster, and stay safer.

ACKNOWLEDGEMENTS D.R.S. and C.K.D. acknowledge financial support from the Brenda A. and Robert M. Aiken Strategic Initiative. The authors acknowledge collaboration between Case Western Reserve University, University Hospitals, and the Cleveland Clinic.

AUTHOR CONTRIBUTIONS D.R.S. wrote and edited the manuscript. R.T.L, J.E.V, C.A.Z, C.K.D, J.R.R. and C.M.A contributed heavily to the editing of the manuscript.

ADDITIONAL INFORMATION Competing interests: The authors declare no competing interests.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

REFERENCES 1. Li, R. T. et al. Wearable performance devices in sports medicine. Sports Health 8,

74–78 (2016). 2. Wisbey, B., Montgomery, P. G., Pyne, D. B. & Rattray, B. Quantifying movement

demands of AFL football using GPS tracking. J. Sci. Med. Sport 13, 531–536 (2010).

3. Seshadri, D. R., Drummond, C., Craker, J., Rowbottom, J. R. & Voos, J. E. Wearable devices for sports: new integrated technologies allow coaches, physicians, and trainers to better understand the physical demands of athletes in real time. IEEE Pulse 8, 38–43 (2017).

4. Pantelopoulos, A. & Bourbakis, N. G. A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Trans. Syst. Man Cybern. Part C 40, 1–12 (2010).

5. Choi, J., Ghaffari, R., Baker, L. B. & Rogers, J. A. Skin-interfaced systems for sweat collection and analytics. Sci. Adv. 4, eaar3921 (2018).

6. Bandodkar, A. J., Jeerapan, I. & Wang, J. Wearable chemical sensors: present challenges and future prospects. ACS Sens. 1, 464–482 (2016).

7. Bariya, M., Nyein, H. Y. Y. & Javey, A. Wearable sweat sensors. Nat. Electron. 1, 160–171 (2018).

8. Heikenfeld, J. et al. Wearable sensors: modalities, challenges, and prospects. Lab Chip 18, 217–248 (2018).

9. Amjadi, M., Kyung, K.-U., Park, I. & Sitti, M. Stretchable, skin-mountable, and wearable strain sensors and their potential applications: a review. Adv. Funct. Mater. 26, 1678–1698 (2016).

10. Sun, Y. & Rogers, J. A. Inorganic semiconductors for flexible electronics. Adv. Mater. 19, 1897–1916 (2007).

11. Haddara, Y., Howlader, M., Haddara, Y. M. & Howlader, M. M. R. Integration of heterogeneous materials for wearable sensors. Polymers 10, 60 (2018).

12. Yao, S., Swetha, P. & Zhu, Y. Nanomaterial-enabled wearable sensors for healthcare. Healthc. Adv. Healthc. Mater. 7, 1700889 (2018).

13. Liu, Y. et al. Flexible, stretchable sensors for wearable health monitoring: sensing mechanisms, materials, fabrication strategies and features. Sensors 18, 1–35 (2018).

14. Rogers, J. A., Someya, T. & Huang, Y. Materials and mechanics for stretchable electronics. Science 327, 1603–1607 (2010).

15. Someya, T., Bao, Z. & Malliaras, G. G. The rise of plastic bioelectronics. Nature 540, 379–385 (2016).

16. ClinicalTrials.gov. Atrial Fibrillation Detection Using Garmin Wearable Technology —Full Text View. https://clinicaltrials.gov/ct2/show/NCT03566836. Accessed 6 Sept 2018.

17. Aliamiri, A. & Shen, Y. Deep learning based atrial fibrillation detection using wearable photoplethysmography sensor. In 2018 IEEE EMBS International Con- ference on Biomedical Health Informatics (BHI) 442–445 (2018).

18. Steinberg, B. A. & Piccini, J. P. Screening for atrial fibrillation with a wearable device. JAMA 320, 139–141 (2018).

19. Choi, D.-H., Kim, J. S., Cutting, G. R. & Searson, P. C. Wearable potentiometric chloride sweat sensor: the critical role of the salt bridge. Anal. Chem. 88, 12241–12247 (2016).

20. Emaminejad, S. et al. Autonomous sweat extraction and analysis applied to cystic fibrosis and glucose monitoring using a fully integrated wearable plat- form. Proc. Natl Acad. Sci. USA 114, 4625–4630 (2017).

21. Schazmann, B. et al. A wearable electrochemical sensor for the real-time mea- surement of sweat sodium concentration. Anal. Methods 2, 342–348 (2010).

22. Kwak, Y. H. et al. Flexible glucose sensor using CVD-grown graphene-based field effect transistor. Biosens. Bioelectron. 37, 82–87 (2012).

23. Chen, Y. et al. Skin-like biosensor system via electrochemical channels for noninvasive blood glucose monitoring. Sci. Adv. 3, e1701629 (2017).

24. Domschke, A. M. Continuous non-invasive ophthalmic glucose sensor for dia- betics. Chimia 64, 43–44 (2010).

25. Kim, J., Campbell, A. S. & Wang, J. Wearable non-invasive epidermal glucose sensors: a review. Talanta 177, 163–170 (2018).

26. Munje, R. D., Muthukumar, S. & Prasad, S. Lancet-free and label-free diagnostics of glucose in sweat using Zinc Oxide based flexible bioelectronics. Sens. Actuators B Chem. 238, 482–490 (2017).

27. Wang, J. Electrochemical glucose biosensors. Chem. Rev. 108, 814–825 (2008). 28. Case, M. A., Burwick, H. A., Volpp, K. G. & Patel, M. S. Accuracy of smartphone

applications and wearable devices for tracking physical activity data. JAMA 313, 625–626 (2015).

29. Gastin, P. B., McLean, O., Spittle, M. & Breed, R. V. P. Quantification of tackling demands in professional Australian football using integrated wearable athlete tracking technology. J. Sci. Med. Sport 16, 589–593 (2013).

30. Waldron, M., Worsfold, P., Twist, C. & Lamb, K. Concurrent validity and test-retest reliability of a global positioning system (GPS) and timing gates to assess sprint performance variables. J. Sports Sci. 29, 1613–1619 (2011).

31. Hausler, J., Halaki, M. & Orr, R. Application of global positioning system and microsensor technology in competitive rugby league match-play: a systematic review and meta-analysis. Sports Med. Auckl. NZ 46, 559–588 (2016).

32. 2017 ISAKOS Congress Abstract Does Overexertion Correlate with Increased Injury? The Relationship Between Player Load and Soft Tissue Injury in Professional American Football Players Utilizing Wearable Technology. http://www.isakos.com/ meetings/2017congress/AbstractView?EventID=8912&Date=&courseid=. Accessed 4 May 2017.

33. ESPN.com. Player Tracking Coming to the NHL? It’s Complicated. http://www.espn. com/nhl/story/_/id/22604597 (2018).

34. Ganzevles, S., Vullings, R., Beek, P. J., Daanen, H. & Truijens, M. Using tri-axial accelerometry in daily elite swim training practice. Sensors 17, 1–14 (2017).

35. Mooney, R., Corley, G., Godfrey, A., Quinlan, L. R. & ÓLaighin, G. Inertial sensor technology for elite swimming performance analysis: a systematic review. Sensors 16, 1–55 (2015).

36. Silva, A. S., Salazar, A. J., Borges, C. M. & Correia, M. V. Wearable monitoring unit for swimming performance analysis. In Biomedical Engineering Systems and Technologies, Vol. 273 (eds. Fred, A., Filipe, J. & Gamboa, H.) 80–93 (Springer, Berlin, Heidelberg, 2013).

37. Walker, E. J., McAinch, A. J., Sweeting, A. & Aughey, R. J. Inertial sensors to estimate the energy expenditure of team-sport athletes. J. Sci. Med. Sport 19, 177–181 (2016).

38. Shcherbina, A. et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J. Pers. Med. 7, 3 (2017).

39. Calvert, T. W., Banister, E. W., Savage, M. V. & Bach, T. A systems model of the effects of training on physical performance. IEEE Trans. Syst. Man Cybern. SMC-6, 94–102 (1976).

40. Morton, R. H. Modelling training and overtraining. J. Sports Sci. 15, 335–340 (1997).

41. Micklewright, D., St Clair Gibson, A., Gladwell, V. & Al Salman, A. Development and validity of the rating-of-fatigue scale. Sports Med. Auckl. NZ 47, 2375–2393 (2017).

42. Noakes, T. D. Fatigue is a brain-derived emotion that regulates the exercise behavior to ensure the protection of whole body homeostasis. Front. Physiol. 3, 1–13 (2012).

43. Zając, A., Chalimoniuk, M., Maszczyk, A., Gołaś, A. & Lngfort, J. Central and peripheral fatigue during resistance exercise—a critical review. J. Hum. Kinet. 49, 159–169 (2015).

44. Kirkendall, D. T. Mechanisms of peripheral fatigue. Med. Sci. Sports Exerc. 22, 444–449 (1990).

45. Enoka, R. M. & Duchateau, J. Muscle fatigue: what, why and how it influences muscle function. J. Physiol. 586, 11–23 (2008).

46. Gabbett, T. J. The training—injury prevention paradox: should athletes be training smarter and harder? Br. J. Sports Med. 50, 273–280 (2016).

47. Buchheit, M., Gray, A. & Morin, J.-B. Assessing stride variables and vertical stiffness with GPS-embedded accelerometers: preliminary insights for the monitoring of neuromuscular fatigue on the field. J. Sports Sci. Med. 14, 698–701 (2015).

48. DeMartini, J. K. et al. Physical demands of National Collegiate Athletic Associa- tion Division I football players during preseason training in the heat. J. Strength Cond. Res. 25, 2935–2943 (2011).

D.R. Seshadri et al.

16

npj Digital Medicine (2019) 71 Scripps Research Translational Institute

49. Gabbett, T. J. Quantifying the physical demands of collision sports: does microsensor technology measure what it claims to measure? J. Strength Cond. Res. 27, 2319–2322 (2013).

50. Wellman, A. D., Coad, S. C., Goulet, G. C. & McLellan, C. P. Quantification of competitive game demands of NCAA Division I college football players using global positioning systems. J. Strength Cond. Res. 30, 11–19 (2016).

51. Gabbett, T. J. et al. The athlete monitoring cycle: a practical guide to interpreting and applying training monitoring data. Br. J. Sports Med. 51, 1451–1452 (2017).

52. Hulin, B. T. et al. Spikes in acute workload are associated with increased injury risk in elite cricket fast bowlers. Br. J. Sports Med. 48, 708–712 (2014).

53. White R. Science for Sport. Acute:chronic workload ratio https://www. scienceforsport.com/acutechronic-workload-ratio/ (2017).

54. Murray, N. B., Gabbett, T. J., Townshend, A. D. & Blanch, P. Calculating acute: chronic workload ratios using exponentially weighted moving averages pro- vides a more sensitive indicator of injury likelihood than rolling averages. Br. J. Sports Med. 51, 749–754 (2017).

55. Edwards, S., White, S., Humphreys, S., Robergs, R. & O’Dwyer, N. Caution using data from triaxial accelerometers housed in player tracking units during run- ning. J. Sports Sci. 0, 1–9 (2018).

56. Why GPS Systems Might Miscalculate Athlete Mechanical Load https://imeasureu. com/2018/11/15/why-gps-systems-may-be-miscalculating-your-athletes- mechanical-load/ (IMeasureU, 2018).

57. ESPN.com. How Nickel-sized Tech Helps Eagles Track Wentz’s Recovery. http:// www.espn.com/blog/philadelphia-eagles/post/_/id/25698 (2018).

58. Wang, L. et al. Why do woodpeckers resist head impact injury: a biomechanical investigation. PLoS ONE 6, e26490 (2011).

59. Mannix, R., Meehan Iii, W. P. & Pascual-Leone, A. Sports-related concussions— media, science and policy. Nat. Rev. Neurol. 12, 486–490 (2016).

60. Ropper, A. H. & Gorson, K. C. Concussion. N. Engl. J. Med. 356, 166–172 (2007). 61. Kelly, J. P. et al. Concussion in sports: guidelines for the prevention of cata-

strophic outcome. JAMA 266, 2867–2869 (1991). 62. Giza, C. C. et al. Summary of evidence-based guideline update: evaluation and

management of concussion in sports. Neurology 80, 2250–2257 (2013). 63. Guskiewicz, K. M. et al. Recurrent concussion and risk of depression in retired

professional football players. Med. Sci. Sports Exerc. 39, 903–909 (2007). 64. Guskiewicz, K. M. et al. Cumulative effects associated with recurrent concussion

in collegiate football players: the NCAA Concussion Study. JAMA 290, 2549–2555 (2003).

65. Daneshvar, D. H., Nowinski, C. J., McKee, A. C. & Cantu, R. C. The epidemiology of sport-related concussion. Clin. Sports Med. 30, 1–17 (2011).

66. Stenger, J. M., Lawton, E. A., Wright, J. M. & Ricketts, J. Mouthguards: protection against shock to head, neck and teeth. Basal Facts 9, 133–139 (1987).

67. McCrory, P. Do mouthguards prevent concussion? Br. J. Sports Med 35, 81–82 (2001).

68. Yu, K. J. et al. Bioresorbable silicon electronics for transient spatiotemporal mapping of electrical activity from the cerebral cortex. Nat. Mater. 15, 782–791 (2016).

69. Noggin Sport: Protective Skull Caps for Sports. http://www.nogginsport.com/. Accessed 5 May 2017.

70. Taylor, T. New collar shows promise for concussion prevention. SI.com. Available at: https://www.si.com/edge/2016/06/15/concussion-prevention-technology- qcollar-neck-wearable-football-hockey. Accessed 5 May 2017.

71. Myer, G. D. et al. The effects of external jugular compression applied during head impact exposure on longitudinal changes in brain neuroanatomical and neurophysiological biomarkers: a preliminary investigation. Front. Neurol. 7, 74 (2016).

72. Myer, G. D. et al. Analysis of head impact exposure and brain microstructure response in a season-long application of a jugular vein compression collar: a prospective, neuroimaging investigation in American football. Br. J. Sports Med. 50, 1276–1285 (2016).

73. X2: ICE (X2 Biosystems) https://www.prnewswire.com/news-releases/x2-bio systems-introduces-new-comprehensive-head-impact-management-system- 300277510.html (2017).

74. Rains, B. X2 Biosystems Introduces their Next Generation X-Patch Pro Head Impact Monitor https://www.sporttechie.com/x2-biosystems-introduces-their- next-generation-x-patch-pro-head-impact-monitor/ (SportTechie, 2016).

75. Reynolds, B. B. et al. Practice type effects on head impact in collegiate football. J. Neurosurg. 124, 501–510 (2016).

76. Morrison, M. & Daigle, J. N. A biosensing approach for detecting and managing head injuries in American football. J. Biosens. Bioelectron. 06, (2015).

77. ESPN.com. Could X2’s Skin Patch Detect Concussions? http://www.espn.com/ blog/playbook/tech/post/_/id/3547 (2013).

78. Wu, L. C., Zarnescu, L., Nangia, V., Cam, B. & Camarillo, D. B. A head impact detection system using SVM classification and proximity sensing in an instru- mented mouthguard. IEEE Trans. Biomed. Eng. 61, 2659–2668 (2014).

79. Wu, L. C. et al. In vivo evaluation of wearable head impact sensors. Ann. Biomed. Eng. 44, 1234–1245 (2016).

80. Talbot, D. An impact-sensing skullcap from rebook could help prevent brain injury. MIT Technol. Rev. https://www.technologyreview.com/s/429751/emtech- reebok-952and-mc10-will-launch-an-impact-sensing-skullcap-for-sports/. Acces- sed 31 Mar 2019.

81. New sensor system detects early signs of concussion in real time. ScienceDaily. https://www.sciencedaily.com/releases/2014/05/140501101008.htm. Accessed 31 Mar 2019.

82. Nguyen, K. D., Chen, I. M., Luo, Z., Yeo, S. H. & Duh, H. B. L. A wearable sensing system for tracking and monitoring of functional arm movement. IEEEASME Trans. Mechatron. 16, 213–220 (2011).

83. Amjadi, M., Yoon, Y. J. & Park, I. Ultra-stretchable and skin-mountable strain sensors using carbon nanotubes–Ecoflex nanocomposites. Nanotechnology 26, 375501 (2015).

84. Dagdeviren, C. et al. Conformal piezoelectric systems for clinical and experi- mental characterization of soft tissue biomechanics. Nat. Mater. 14, 728–736 (2015).

85. Barrabi, T. Can this device end major league baseball’s arm injury epidemic? FOXBusiness (2016). http://www.foxbusiness.com/features/2016/05/31/can-this- device-end-major-league-baseballs-arm-injury-epidemic.html. Accessed 5 May 2017.

86. Fleisig, G. S. & Andrews, J. R. Prevention of elbow injuries in youth baseball pitchers. Sports Health 4, 419–424 (2012).

87. Welcome. http://motusglobal.com/. Accessed 5 May 2017. 88. Fleisig, G. S. et al. Differences among fastball, curveball, and change-up pitching

biomechanics across various levels of baseball. Sports Biomech. 15, 128–138 (2016).

89. Laughlin, W., Geronimo, A., Close, K., Holstad, R. & Hansen, B. A case study to examine the relationship between elbow valgus torque in football throwing and baseball pitching. 2.

90. Vibrado Technologies. Home. http://www.vibradotech.com/. Accessed 5 May 2017.

91. Woo, S. L.-Y., Wu, C., Dede, O., Vercillo, F. & Noorani, S. Biomechanics and anterior cruciate ligament reconstruction. J. Orthop. Surg. 1, 2 (2006).

92. Dowling, A. V., Favre, J. & Andriacchi, T. P. A wearable system to assess risk for anterior cruciate ligament injury during jump landing: measurements of tem- poral events, jump height, and sagittal plane kinematics. J. Biomech. Eng. 133, 071008–071008–7 (2011).

93. Strath, S. J. et al. Evaluation of heart rate as a method for assessing moderate intensity physical activity. Med. Sci. Sports Exerc. 32, S465–S470 (2000).

94. Terbizan, D. J., Dolezal, B. A. & Albano, C. Validity of seven commercially avail- able heart rate monitors. Meas. Phys. Educ. Exerc. Sci. 6, 243–247 (2002).

95. Komodo A. I. O. Smart sleeve gets serious about heart rate monitoring. Wareable https://www.wareable.com/fitness-trackers/aio-smart-sleeve-specs-price- release-date-2547. Accessed 4 May 2017.

96. Jabra Sport Pulse review. Wareable https://www.wareable.com/headphones/ jabra-sport-pulse-review. Accessed 4 May 2017.

97. Johnstone, J. A., Ford, P. A., Hughes, G., Watson, T. & Garrett, A. T. BioharnessTM

multivariable monitoring device: Part. I: validity. J. Sports Sci. Med. 11, 400–408 (2012).

98. Kim, J.-H., Roberge, R., Powell, J. B., Shafer, A. B. & Williams, W. J. Measurement accuracy of heart rate and respiratory rate during graded exercise and sustained exercise in the heat using the Zephyr BioHarnessTM. Int. J. Sports Med. 34, 497–501 (2013).

99. Dhar, P. et al. Autonomic cardiovascular responses in acclimatized lowlanders on prolonged stay at high altitude: a longitudinal follow up study. PLoS ONE 9, 1–11 (2014).

100. Wang, R. et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2, 104–106 (2017).

101. Gillinov, S. et al. Variable accuracy of wearable heart rate monitors during aerobic exercise. Med. Sci. Sports Exerc. 49, 1697–1703 (2017).

102. Stahl, S. E., An, H.-S., Dinkel, D. M., Noble, J. M. & Lee, J.-M. How accurate are the wrist-based heart rate monitors during walking and running activities? Are they accurate enough? BMJ Open Sport Exerc. Med. 2, e000106 (2016).

103. Lee, S. P. et al. Highly flexible, wearable, and disposable cardiac biosensors for remote and ambulatory monitoring. Npj Digit. Med. 1, 2 (2018).

104. Hertzman, A. B. The blood supply of various skin areas as estimated by the photoelectric plethysmograph. Am. J. Physiol.-Leg. Content 124, 328–340 (1938).

105. Kabir, M. M., Perez-Alday, E. A., Thomas, J., Sedaghat, G. & Tereshchenko, L. G. Optimal configuration of adhesive ECG patches suitable for long-term mon- itoring of a vectorcardiogram. J. Electrocardiol. 50, 342–348 (2017).

106. Tereshchenko, L. G. et al. Analysis of speed, curvature, planarity and frequency characteristics of heart vector movement to evaluate the electrophysiological

D.R. Seshadri et al.

17

Scripps Research Translational Institute npj Digital Medicine (2019) 71

substrate associated with ventricular tachycardia. Comput. Biol. Med. 65, 150–160 (2015).

107. Baggish, A. L. et al. Sports cardiology: core curriculum for providing cardiovas- cular care to competitive athletes and highly active people. J. Am. Coll. Cardiol. 70, 1902–1918 (2017).

108. Vrsansky, N. Case Western student researching device to Help Student Athlete Performance. http://www.cleveland19.com /2018/12/10/case-western-student- researching-device-help-athletic-performances/. Accessed 18 Dec 2018.

109. Lee, S. M. et al. Self-adhesive epidermal carbon nanotube electronics for tether- free long-term continuous recording of biosignals. Sci. Rep. 4, 1–9 (2014).

110. Yeo, W.-H. et al. Multifunctional epidermal electronics printed directly onto the skin. Adv. Mater. 25, 2773–2778 (2013).

111. Hu, D., Cheng, T. K., Xie, K. & Lam, R. H. W. Microengineered conductive elas- tomeric electrodes for long-term electrophysiological measurements with consistent impedance under stretch. Sensors 15, 26906–26920 (2015).

112. Lee, J.-W., Yun, K.-S., Lee, J.-W. & Yun, K.-S. ECG monitoring garment using conductive carbon paste for reduced motion artifacts. Polymers 9, 439 (2017).

113. SportTechie. Smart Textile Company Kymira introduces cardiac monitoring T-shirt (2018). https://www.sporttechie.com/smart-textile-company-kymira- cardiac-monitoring-tshirt-athletes/. Accessed 18 Dec 2018.

114. Anderson, D. P. & Engineer, B.-A. The limitations of training with heart rate and the crucial information muscle oxygenation can offer you. 4 (2017).

115. Seiler, K. S. & Kjerland, G. Ø. Quantifying training intensity distribution in elite endurance athletes: is there evidence for an ‘optimal’ distribution? Scand. J. Med. Sci. Sports 16, 49–56 (2006).

116. Esteve-Lanao, J., Foster, C., Seiler, S. & Lucia, A. Impact of training intensity distribution on performance in endurance athletes. J. Strength Cond. Res. 21, 943–949 (2007).

117. Farzam, P., Starkweather, Z. & Franceschini, M. A. Validation of a novel wearable, wireless technology to estimate oxygen levels and lactate threshold power in the exercising muscle. Physiol. Rep. 6, e13664 (2018).

118. Yodh, A. & Chance, B. Spectroscopy and imaging with diffusing light. Phys. Today 48, 34 (2008).

119. Hamaoka, T., McCully, K. K., Niwayama, M. & Chance, B. The use of muscle near- infrared spectroscopy in sport, health and medical sciences: recent develop- ments. Philos. Trans. A 369, 4591–4604 (2011).

120. Yu, G. et al. Time-dependent blood flow and oxygenation in human skeletal muscles measured with noninvasive near-infrared diffuse optical spectro- scopies. J. Biomed. Opt. 10, 024027 (2005).

121. Joyner, M. J. & Casey, D. P. Regulation of increased blood flow (hyperemia) to muscles during exercise: a hierarchy of competing physiological needs. Physiol. Rev. 95, 549–601 (2015).

122. Chance, B., Dait, M. T., Zhang, C., Hamaoka, T. & Hagerman, F. Recovery from exercise-induced desaturation in the quadriceps muscles of elite competitive rowers. Am. J. Physiol. 262, C766–C775 (1992).

123. Whoop. Performance optimization system|wearable fitness devices. https://whoop.com/. Accessed 4 May 2017.

124. Sekiguchi, Y. et al. Relationships between resting heart rate, heart rate variability and sleep characteristics among female collegiate cross-country athletes. J. Sleep Res. 0, e12836 (2019).

125. VanHelder, T. & Radomski, M. W. Sleep deprivation and the effect on exercise performance. Sports Med. Auckl. NZ 7, 235–247 (1989).

126. Martin, B. J. Sleep loss and subsequent exercise performance. Acta Physiol. Scand. Suppl. 574, 28–32 (1988).

127. Fatigue Science Public Website. fatiguescience. 5 areas sleep has the greatest impact on athletic performance. https://www.fatiguescience.com/blog/5-ways- sleep-impacts-peak-athletic-performance/ (2015)

128. Milewski, M. D. et al. Chronic lack of sleep is associated with increased sports injuries in adolescent athletes. J. Pediatr. Orthop. 34, 129–133 (2014).

129. Studies link fatigue and sleep to major league baseball (MLB) performance and career longevity. https://www.sleepfoundation.org/sleep-news/studies-link- fatigue-and-sleep-major-league-baseball-mlb-performance-and-career- longevity. Accessed 9 Jan 2019.

130. Peak Performance. Sleep deprivation: how does it affect performance? https:// www.peakendurancesport.com/endurance-injuries-and-health/endurance- health-and-lifestyle/lack-sleep-can-effect-performance/ (2017).

131. Lee, J.-M. et al. Comparison of wearable trackers’ ability to estimate. Sleep. Int. J. Environ. Res. Public. Health 15, 1265 (2018).

132. Rosenberger, M. E., Buman, M. P., Haskell, W. L., McConnell, M. V. & Carstensen, L. L. Twenty-four hours of sleep, sedentary behavior, and physical activity with nine wearable devices. Med. Sci. Sports Exerc. 48, 457–465 (2016).

133. Marino, M. et al. Measuring sleep: accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep 36, 1747–1755 (2013).

134. Armon C., Johnson K. G., Roy A. & Nowack W. J. Polysomnography: overview, parameters monitored, procedures. https://emedicine.medscape.com/article/ 1188764-overview (2018).

135. Mantua, J., Gravel, N. & Spencer, R. M. C. Reliability of sleep measures from four personal health monitoring devices compared to research-based actigraphy and polysomnography. Sensors 16, 646 (2016).

136. Shrier, I. Strategic Assessment of Risk and Risk Tolerance (StARRT) framework for return-to-play decision-making. Br. J. Sports Med. 49, 1311–1315 (2015).

137. Creighton, D. W., Shrier, I., Shultz, R., Meeuwisse, W. H. & Matheson, G. O. Return- to-play in sport: a decision-based model. Clin. J. Sport Med. 20, 379–385 (2010).

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing,

adaptation, distribution and reproduction in anymedium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons. org/licenses/by/4.0/.

© The Author(s) 2019

D.R. Seshadri et al.

18

npj Digital Medicine (2019) 71 Scripps Research Translational Institute