hyperactivity disorder mobile apps: A systematic review

International Journal of Medical Informatics 138 (2020) 104133

Contents lists available at ScienceDirect

International Journal of Medical Informatics journal homepage: www.elsevier.com/locate/ijmedinf

Review article

Attention-deficit/ hyperactivity disorder mobile apps: A systematic review a,b

Costina Ruxandra Păsărelu

c,d

, Gerhard Andersson , Anca Dobrean

a,b,

*

T

a

Department of Clinical Psychology and Psychotherapy, Babeș-Bolyai University, Cluj-Napoca, Romania The International Institute for the Advanced Studies of Psychotherapy and Applied Mental Health, Cluj-Napoca, Romania Department of Behavioral Sciences and Learning, Linköping University, Linköping, Sweden d Department of Clinical Neuroscience, Division of Psychiatry, Karolinska Institute, Stockholm, Sweden b c

ARTICLE INFO

ABSTRACT

Keywords: Attention-deficit/hyperactivity disorder Apps Mobile Children Parent

Background: Attention-Deficit/ Hyperactivity Disorder (ADHD) is a prevalent condition in children and adolescents. Although there are pharmacological and non-pharmacological treatments for this disorder, barriers in accessing evidence-based treatments are still a major problem. Digital health interventions are promising for multiple mental health problems. Recent years have brought an increase in the number of existing mobile apps designed for the management of ADHD. The aim of this study was to systematically review the existing mobile apps designed for ADHD in terms of general characteristics, empirical support for their development and efficacy/ effectiveness, and to describe the content and design of the four most downloaded ADHD apps. Method: We conducted systematic searches on iTunes/iOS (Apple App Store), Google Play and the National Health Service Health Apps Library up to May 2017 and checked for changes in March and September 2019. We included those apps that were designed for ADHD, target assessment, treatment, or both, were in English and were functional. We identified 355 apps in the virtual stores, out of which we included 109 apps in the present systematic review. For each app we extracted the following information: target population, developer, price, number of downloads, ratings, privacy, available language other than English, category, purpose and empirical support. A second search was conducted in literature databases up to September 2019: PsycINFO, Pubmed, Scopus, Web of Science, Cochrane database. Results: We found 109 ADHD apps in the virtual stores available to both young and adult populations, developed for children, adolescents, parents, teachers, and professionals. Most of the apps included are designed for treatment, or ADHD symptoms assessment, however, we found several apps designed for both assessment and treatment purposes. Very few apps contained information regarding their development and none contained information regarding evidence for its efficacy/ effectiveness. Four apps were the most downloaded, with 10,000 (three apps) to 100,000–500,000 (one app) downloads. Out of 51 papers identified through systematic searches in the literature, we identified only one study that met our inclusion criteria, however, this study was published in 2018, outside of the 2017 app store search, therefore, there is no overlap between evidence in the literature and apps on the market. Conclusions: Given the large proliferation of mental health apps and their potential benefits in terms of dissemination and costs, future research needs to be conducted in order to establish the safety and efficacy of ADHD apps available on the commercial market.

1. Introduction Attention-deficit/hyperactivity disorder (ADHD) is a prevalent condition in youths and adults [1,2] and is associated with many negative consequences for both the individual suffering and his/her family [3,4]. According to the Diagnostic and Statistical Manual of Mental Disorders - 5th edition (DSM-5), core symptoms of ADHD include inattention, hyperactivity, and impulsivity [5]. Available treatments for ADHD exist, and the National Institute for Health and Care

⁎

Excellence guidelines [6] recommend pharmacological treatment (e.g., stimulant, non-stimulant medication), psychological treatment, combined pharmacological and psychological treatment as a function of age (e.g., children, adults) and impairment. Effect size of standard treatments is moderate for behavioral interventions (standardized mean differences = .40). However, when blinded assessors rated the outcomes, these effects became nonsignificant [7]. Regarding pharmacotherapy, results of a network meta-analysis indicate methylphenidate as a first-choice pharmacological treatment in the short-term for youths

Corresponding author at: Babeș-Bolyai University, Republicii St., No. 37, Postal code: 400015, Romania. E-mail address: [email protected] (A. Dobrean).

https://doi.org/10.1016/j.ijmedinf.2020.104133 Received 11 April 2018; Received in revised form 23 February 2020; Accepted 23 March 2020 1386-5056/ © 2020 Elsevier B.V. All rights reserved.

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

with ADHD (large effect sizes compared to placebo) and amphetamines for adults with ADHD [8]. Furthermore, a large number of children do not respond to pharmacological treatment (approximately 30 %; for a review of clinical and neurobiological predictors of treatment-refractory ADHD see ref [9]) or have adverse reactions to it. Developments in mental health delivery (e.g., Internet-delivered therapy) have brought important advantages, such as access to treatment in remote areas, lower costs as compared to face-to-face treatment, and numerous other benefits for both patients (e.g., they can review therapy contents whenever they want, they can receive therapist support more rapidly as compared to their weekly therapy sessions) and therapists (e.g., saved resources such as time) [10]. Digital innovations in the form of computerized interventions, Internet-delivered interventions, gamified interventions or mental health apps have become new promising paths in dealing with mental health problems [11], as they have the potential to overcome barriers in the dissemination of evidence-based interventions [12]. For instance, Internet interventions are efficient for adult anxiety and depressive disorders [13], as well as for youth populations with mental health problems [14]. However, as compared to interventions delivered for other mental health problems (e.g., anxiety disorders, depressive disorder), research on the effectiveness of digital health interventions for patients with ADHD is far more under-represented. Internet-delivered cognitive behavioral treatment was effective in a sample of adults with ADHD [15], with a large effect size between groups for the inattention outcomes (Cohen’s d = 1.21). Another intervention delivered via smartphones based on organizational skills training helped adults with ADHD to gain more structure in their lives [16]. Regarding research conducted for parents or children with ADHD, we have found in the literature preliminary support for a serious game intervention and an online intervention for parents. Namely, a serious game for iPad designed for ADHD was feasible, however, the sample was very small (six children with ages between 8 and 12 years old [17]). Data coming from a randomized controlled trial showed that an online intervention for parents of children with ADHD symptoms was effective in reducing child ADHD symptoms and led to improvement in several parent-related outcomes [18]. Mental health apps could be cost-effective [19], associated with both direct and indirect effects on youths, parents, and professionals. A large number of apps are available for various populations such as adults or young populations [20], with different mental health problems, such as bipolar disorder [21,22], obsessive-compulsive disorders, posttraumatic stress, anxiety disorders and affective disorders [23], social anxiety [24], depression [25], and suicide prevention (for a review see [26]). The efficacy of mental health apps for individuals suffering from ADHD has only recently been tested. We found several articles that used a qualitative or quantitative approach. Two studies conducted by the same team used qualitative analysis to investigate the suitability of the top ten ADHD apps (5 from Google Play and 5 from iTunes) [27,28]. One of the studies investigated the views of parents and clinicians working with parents on the suitability of the apps, while the other study investigated the views of ADHD-diagnosed children and of clinicians working with children on the suitability and usability of the apps. Research is scarce regarding the effectiveness of mobile apps designed for ADHD, with several pilot studies conducted with small samples, case studies presented as conference papers or dissertations presenting initial research for several apps developed either for assessment purposes, combined assessment and intervention, or intervention alone. Regarding assessment apps, in a pilot study conducted on 11 healthy adults, Snappy App, which incorporates a Continuous Performance Test to assess ADHD symptoms, had adequate user acceptability [29] and significant associations were found with self-reported impulsivity and sensor data on activity. Preliminary evidence of WHAAM app is presented through a conference paper. This app designed for parents, teachers, and clinicians of children with ADHD is based on behavioral therapy, which helps them with the observation of behaviors displayed in various settings [30].

For apps designed to be used both for assessment and intervention, data coming from another a small pilot study indicated that an iPad web-based app, called iSelfControl, designed to monitor and improve classroom behavioral management in a sample of children with school difficulties related to attention deficits and low self-control [31] could identify discrepancies in evaluation between children and teachers. The same study showed increases in children’s self-awareness, as well as a high satisfaction with the intervention received, as 70 % of the students reported that the app helped them and that they liked it. For mobile apps designed for intervention, we found preliminary evidence for three mobile apps on outcomes such as behavior, academic functioning, or reading motivation. For the first category, we identified in the literature a report on the app ADHD Trainer [32] presenting a case study on a child diagnosed with ADHD. The content of the app was cognitive training, and it was delivered together with medication. Results showed changes from pre-treatment to post-treatment assessments on behavior and academic functioning. The development and preliminary efficacy of an app designed to help children with ADHD to read by increasing their motivation were described in a thesis [33]. Other mobile apps were designed to improve physical activity. For instance, an intervention combining the Fitbit mHealth app, the Fitbit Flex wristband and joining a Facebook group was feasible and improved physical activity in adolescents (n = 11) with ADHD [34]. 1.1. Rationale for the systematic review Given the potential advantages that digital innovations have in the treatment of mental health problems, the aim of the present study was to conduct a systematic review of the available mental health apps for ADHD. Existing research conducted on mobile apps developed for mental health conditions showed that their number is increasing, and preliminary results indicated favorable outcomes. Moreover, a recent systematic review of existing technologies for children and adolescents with ADHD shows that such interventions could be useful in the management of ADHD symptoms [31]. Therefore, given the potential benefits of mobile technologies in the treatment of ADHD in children, adolescents, and adults, we aimed to answer three main research questions regarding the ADHD apps identified: (a) What are the general characteristics of the ADHD apps available in the commercial stores (Apple App Store and Google Play), the National Health Service Health Apps Library and literature (PsycINFO, PubMed, Web of Science, Scopus, Cochrane database)? (b) What is the empirical evidence underlying ADHD apps? In order to answer this question, we aimed to investigate evidence described in the content of the apps (how was the app developed and tested?), as well as coming from the research conducted so far. (c) What are the main characteristics of the most downloaded ADHD apps? Such a review could have many important implications for the clinicians and psychotherapists who wonder whether they could use the latest advances in technologies (e.g., apps) in their practice, whether it is safe to use such apps or they could do more harm, what the support of the existing apps is, or which apps they should use according to the empirical evidence. Also, this study can have important implications for users, namely parents and teachers who search for information on ADHD online or via mobile apps. Based on such a review, parents and teachers can make a more informed decision regarding which apps are safe to use according to their empirical support and which apps have not been tested so far. Therefore, even though it may be tempting to use mobile apps in the assessment or management of ADHD, app users should first question their support. Finally, as our review analyzes the content and empirical evidence of ADHD apps, this could have important implications for researchers interested in the treatment of ADHD, as it provides research directions that still need to be investigated. 2

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

2. Material and methods

following information was extracted:

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement [35]. We conducted two systematic searches, one in the marketplace and one in the literature databases. In order to identify the relevant apps in the commercially available stores and app libraries, systematic searches were conducted on iTunes/ iOS (Apple App Store), Android (Google Play) and the National Health Service (NHS) Apps Library. The search was conducted in May 2017. Both virtual stores, as well as NHS Apps Library were checked again in March and September 2019 for changes. The search in the literature was conducted in PsycINFO, PubMed, Web of Science, Scopus, Cochrane database in September 2019. Keyword terms used for systematic searches in virtual stores were: ADHD, attention-deficit hyperactivity disorder, hyperactivity. The search terms selected were used as a disjunctive query, with the inclusion of the apps containing at least one of the terms mentioned. Search terms used in the literature review were related to ADHD (“attention-deficit/ hyperactivity disorder” OR “hyperactivity” OR “inattention” OR “attention deficit”) and mobile applications (“mobile app” OR “mHealth” OR “smartphone app”). We searched for titles, abstracts and keywords. Two independent searches were conducted in all three databases (the first author - author name removed for blind review and a Ph.D. student from the same department). The inclusion criteria for the apps coming from the marketplace were:

a Target population. We extracted from the app the age group (children or adolescents, adults), or whether the apps are delivered for specific groups, such as parents, teachers or professionals. b Type of developer. We coded whether the app was developed by one or more individuals or by a private company or a university. c Cost. We extracted information on app price, namely for each app we rated whether it was free of charge or paid, in which case we also extracted the exact cost in dollars. d The number of downloads. This information was available only for apps found in Google Play, as iTunes does not offer such an indicator. We noted the exact number of downloads where available. e User ratings. We extracted from Google Play and iTunes/iOS (Apple App Store) stores the number of stars available for each app, ranging from one to five stars, and the total number of reviewers of the app. f Confidentiality. We coded whether the app mentioned a Privacy section and whether this was available. g App category. We extracted the name of the category in which the app was included on iTunes or Google Play, namely: Medical, Health and Fitness, Lifestyle, Education, Productivity, References, Games or Parenting. h Available language in addition to English. We extracted the number and the name of the available translations in which the app was available. i The main purpose of the app. According to the app content and aims, we included the apps in one of the three categories: assessment, treatment, or mixed assessment and treatment. Besides the main purpose of the app, for those apps that referred to ADHD treatments, namely apps included in the second and third category, we coded the specific type of treatment offered. Therefore, apps were included in one of these seven categories: pharmacological treatment (tracking and monitoring adherence), organizational skills training, cognitive training, neurofeedback, hypnosis, psychoeducation, and music therapy. j Empirical evidence underlying the app (theory and/or treatment package). We coded this characteristic according to the information provided in the app.

a) App description should include that it is an app designed for ADHD; b) To include information about psychological or pharmacological treatment or psychological assessment for ADHD as a main aim of the app. Only those apps that were designed to assess or treat ADHD symptoms were included, while the apps that were only tangentially related to ADHD were excluded (e.g., aimed general strategies, but the app was not designed for ADHD assessment nor treatment); c) No comparison needed; d) To contain information in English; e) To be functional, namely to be accessible to be downloaded and to work properly. Apps were excluded in case: i) they were apps designed to improve general working memory performance, intelligence, ii) were only marginally related to ADHD (e.g., were not designed specifically for ADHD, but could be used by anyone for organizational skills training, for instance, apps that allow to plan and organize the daily schedule). The inclusion criteria for the papers coming from literature were:

2.2. Data analysis Descriptive statistics were computed in the Statistical Package for the Social Sciences (SPSS) version 20 [36]. First, characteristics of the included ADHD apps were extracted in an Excel file and then imported in SPSS. In order to assess inter-rater reliability, we computed Cohens’s kappa, which evaluates the level of agreement between raters and uses corrections for agreement expected by chance [37]. Agreement between the two authors who independently searched the three databases was high (96 %). The level of agreement between the two authors who included the apps in the three categories (assessment, treatment, mixed) was high (Cohen’s kappa = .92), according to the guidelines for Kappa interpretation, where values between .81 and 1 reflect almost perfect agreement [38]. Disagreements were discussed until consensus was reached.

a) Population: individuals diagnosed with ADHD or with ADHD symptoms; b) Intervention: mobile application developed specifically for ADHD; c) Comparator/ control: any comparison; d) Outcome: include at least one outcome related to ADHD (changes in inattention, hyperactivity, impulsivity), or other behavioural outcomes; e) Study type: feasibility study, pilot study, randomized control trial; f) Other: studies were published in peer-reviewed journals, in English.

3. Results

Papers were excluded if the participants were healthy or ADHD was not their primary diagnosis, the intervention was not delivered via a mobile app (e.g., Internet intervention, web-based interventions, SMSs), design was single case or case study, reported outcomes were related to app development (usability), or were systematic reviews or theoretical papers, conference abstracts, letter to editors.

3.1. Selection Apps identified in the marketplace and digital app libraries Fig. 1 illustrates the systematic search procedure in the databases selected. Electronic searches resulted in 355 apps (255 apps identified in Google Play, 100 apps identified in iTunes, 0 apps identified in the NHS apps database). After removing duplicates, namely apps that appeared in both stores (19 apps) and apps that appeared on different searches using each keyword, we analyzed 211 apps. The next step was to exclude 66 apps because the content was not specifically designed for

2.1. Data extraction Two independent reviewers (CRP and AD) categorized the ADHD apps according to several characteristics established a priori. The 3

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

Fig. 1. Apps review flow chart.

ADHD. For instance, we excluded one app that provides important dates and news regarding scientific events such as conferences, a radio app where participants can listen to audio content on various topics and an app providing dictionary definitions of multiple conditions. We also excluded magazines that only tangentially provided information about ADHD and a cookbook covering recipes based on a gluten-free and milk-free diet for children with autism and ADHD. These apps were excluded as the information provided could not fit into a category (assessment or treatment). Finally, 109 apps identified in virtual stores were included in the review. Systematic searches in PsycINFO, PubMed, Web of Science, Scopus and Cochrane database lead to the identification of 51 papers. After we removed duplicates, we screened the title and abstracts of 25 relevant papers. We selected eight articles to read full-text, as the rest of the papers were either: systematic reviews (n = 3) or theoretical papers (n = 3), qualitative studies with no outcomes related to ADHD (n = 3), scale development (n = 1), not delivered for ADHD primary problem (n = 2), not related to ADHD (n = 1), case report (n = 1), letter to the editor (n = 1), or theses/ dissertations (n = 2). Out of the eight articles selected for full-text read, two studies did not meet our inclusion criteria as they were conference papers, one article described the app development process, three studies were feasibility studies with no ADHD-related outcomes measured and in one study the intervention was not an app developed specifically for ADHD. We found only one study that met all our inclusion criteria. Weisman et al. [39] investigated the efficacy of the iCON™ mobile app, a smartphone application developed to improve medication adherence

for children with ADHD by facilitating communication between the treating physician and the parents. The app allows parents to receive daily notifications that help them remind their children to adhere to the pharmacological treatment, as well as to complete behavioral assessments on child symptoms, functionality, and side effects. ADHD psychoeducation through video and text-based materials is also available in the app. Participants were 39 Israeli children diagnosed with ADHD, mostly boys, aged between 6 and 16, recruited from a child and adolescent psychiatry outpatient unit. Participants were randomly allocated either to the experimental (prompted to use an app for medication adherence) or to a control condition (treatment-as-usual, no app). Results of this study indicated significant differences between the experimental group and the control group in adherence to stimulants (measured as overall pill counts) and ADHD symptom severity. 3.2. General description of the apps identified in the marketplace We grouped the 109 included apps in several categories: children, parents, teachers, professionals, or adults depending on the target group for which the app appeared to be intended. Table 1 illustrates the number of apps developed for each category of users and other general characteristics such as the developer, costs, number of downloads, user ratings and confidentiality policy. More than half of the apps included (57 %) cost between $0.99 and $14.99. The mean price was $4.48 (standard deviation = 2.91). Another indicator we considered was the number of downloads available for those apps alone coming from Google Play store; this estimate 4

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

German, Italian, Japanese, Korean, Russian, Serbian, Chinese, Urdu, and Turkish.

Table 1 General characteristics of the ADHD apps found in virtual stores, N = 109.

Age groups Not specified Adults Children or adolescents Parents Teachers Professionals Developer Individual Company or university Cost Free Price Number of downloads Unknown 1−10 > 10 > 100 > 1000 > 10,000 > 100,000 User ratings No rating provided 1−2.9 3−3.9 4−4.9 5 Confidentiality Privacy policy available No privacy policy

N apps

%

43 33 26 31 7 8

39.45 30.27 23.85 28.44 6.42 7.34

16 93

14.68 85.32

47 62

43.12 56.88

55 7 13 14 16 3 1

50.46 6.42 11.93 12.84 14.68 2.75 0.92

62 11 19 11 6

56.88 10.09 17.43 10.09 5.51

64 45

58.72 41.28

3.5. Main purpose of the app We grouped the ADHD apps included according to their intended purpose as mentioned in the app’s claim and its content. More specifically, we grouped ADHD apps in three categories: ADHD symptom assessment, ADHD treatment, and a mixed assessment-treatment category (see Table 3). 3.5.1. Assessment apps Three apps in this category were specifically intended to be used by professionals, while the remaining apps could be used as self-report assessments of ADHD symptoms. Regarding the instruments used to assess the ADHD symptomatology, only a few apps reported the name of the instrument used. In two apps the Swanson, Nolan, and Pelham-IV Questionnaire (SNAP-IV; [40]) was used. Three apps mention that the questions used to assess ADHD symptomatology were derived from a World Health Organization (WHO) approved questionnaire, and only two apps mention the name of the WHO instrument, namely the WHO Adult ADHD Self-Report Scale (ASRS; [41]). One app was based on the Diagnostic Interview for ADHD in Adults (DIVA 2.0; [42]), and one app used the Jasper/Goldberg Adult ADD Questionnaire. Finally, the description of one ADHD assessment app mentioned that the questions were based on the DSM-5 [5] criteria. 3.5.2. Treatment apps In the categorization of treatment strategies provided, we identified several major treatment modalities, such as tracking and monitoring ADHD pharmacological treatment, organizational skills training, psychoeducation, hypnosis, cognitive training, neurofeedback, or music therapy. Table 3 illustrates ADHD apps that are based on each of these treatment modalities. Most of the ADHD apps included in this review were based on music therapy (28 %), followed by ADHD psychoeducation (24 %), organizational skills training (18 %) and tracking and monitoring the pharmacological treatment for ADHD (13 %). Fewer apps offered neurofeedback training (8 %), cognitive training (4 %) and hypnosis (5 %). Apps based on music therapy aimed to improve ADHD symptoms through listening to various music genres: electronic, rock, classic, jazz, hip hop, retro, or country music. Apps based on psychoeducation offered information about ADHD etiology and treatment. One app offered information about current research conducted on ADHD. These psychoeducation apps are mostly based on written material, however, several apps also contained video and graphical content. Apps based on organizational skills training aimed to develop skills in ADHD youths and adults through time management, goals setting and reminders,

Note. ADHD = Attention-deficit/hyperactivity disorder, N = number of apps.

varied between 1–5 and 100,000–500,000 downloads. User ratings were between 1 and 5 stars, with most of the apps having ratings between 3 and 3.9 stars (17 %). These results should be interpreted with caution, as most of the apps were rated by few people, ranging from 1 (n = 7 apps) to 376 reviewers (n = 1 app). 3.3. Category Table 2 depicts the available ADHD apps according to each category (Medical, Health & Fitness, Lifestyle, Education, Productivity, Reference, Games, Parenting). 3.4. Available language Most of the apps were available in English only (91 %), with a few apps being available in both English and other translations (9%). These translations varied from two (e.g., both English and German, 0.9 %; English and Spanish 0.9 %), three (e.g., English., Portuguese and Spanish; 0.9 % or English, Dutch and German; 0.9 %) to nineteen (0.9 %) available translations into languages such as Danish, German, Norwegian, Portuguese, Spanish, Swedish, Bosnian, Croatian, French,

Table 3 Characteristics of ADHD apps found in virtual stores by their main purpose, N = 109.

Table 2 ADHD apps from virtual stores by app type, N = 109. Category

N apps

%

Medical Health & Fitness Lifestyle Education Productivity Reference Games Parenting

29 27 31 10 5 4 2 1

26.6 24.8 28.4 9.2 4.6 3.7 1.8 0.9

Main purpose of the app Assessment Treatment Both treatment and assessment Type of treatment Tracking and monitoring ADHD pharmacological treatment Organizational skills training Cognitive training Neurofeedback Hypnosis Psychoeducation Music therapy

Note. ADHD = Attention-deficit/hyperactivity disorder, N = number of apps.

N apps

%

13 85 11

11.92 77.98 10.10

12 17 4 8 5 23 27

12.50 17.71 4.17 8.33 5.21 23.96 28.12

Note. ADHD = Attention-deficit/hyperactivity disorder, N = number of apps. 5

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

homework planning, rewards. One app was available for teenage drivers and offered reminders to help novice ADHD adolescents to organize their actions better before driving. It also provided text messages to the caregiver about the start and end of the trip, notifications to remind ADHD drivers to adhere to their medical treatment, questions about activities (eating, radio listening) that can appear while driving. In the category of apps designed to track and monitor ADHD treatment, we included those apps that aimed to improve pharmacological treatment adherence. These apps supplied reminders for medication intake through notifications, allowed symptom and treatment monitoring through several graphical depictions of patients’ evolution across time and stored patients’ medication history. Few of these apps facilitated direct communication with the patients’ physicians. We included those apps that aimed to improve attention, working memory and inhibition in the category of cognitive training interventions. Apps based on cognitive training principles were designed as games in which children or adults had to perform several tasks, while their difficulty increases with each level performed. Apps that were based on hypnosis were presented as audio contents in which relaxation suggestions are induced on a music background, or the participant receives several positive affirmations or subliminal messages aimed to improve performance, without relaxation suggestions and music. An example of subliminal message is: ” Mommy and I are one, every day I am better and better, every day I am more confident, I am loved, I am smart, I am worthy, I am confident, I have confidence in me, I am worthy to be that which I want to be, every day I have more confidence” [43]. In order to use those apps based on hypnosis, participants must connect their headphones to their smartphones. We included in the category of apps based on neurofeedback treatment those apps that aimed to improve participants’ brainwaves. Several brain regions that these apps aim to stimulate are the central sulcus, the precentral gyrus, the middle frontal gyrus, the dorsolateral prefrontal cortex, the inferior parietal lobule, the basal ganglia, the cerebellum, the anterior cingulate and the pineal gland.

(b) Are the functional impairments in several areas (social, academic, occupational) also assessed? (c) Are the settings in which difficulties occur (e.g., at least two settings) also investigated? (d) Is symptom criteria adjusted to age changes? (2) The support provided on the psychometric properties of the instruments used (data on validity, reliability, cut-off points and norms for interpretation of results). Therefore, as regards ADHD apps designed for screening/ assessment, we identified several promising apps that could be used for: - Self-assessment of ADHD: ADHD Self Test [45] is an app designed for adult ADHD self-assessment; it provides five categories of score interpretation (unlikely, borderline, mild, moderate, high probability of ADHD). - Other-assessment: an app that involves multiple informants is ADHD Test [46], which is designed for parent rating of ADHD symptoms and has 18 Yes/ No questions. The app description mentions that it is in accordance with the DSM-5 criteria. It has 3 tests, one for hyperactivity (9 items from DSM-5), one for inattention (9 items) and one for ADHD (all 18 items). The interpretations in this test are given in percentage points, that is, the results represent a certain percentage of ADHD items. The Attention Deficit ADHD Test [47] is designed for parents to rate their child’s symptoms as well as their own symptoms. ADHD Medtest [48] is an app for assessment that is designed to be used by parents and teachers to rate ADHD symptoms (attention, concentration, impulsivity). It also contains an attention test for the child. - Clinician assessment: Diva 2.0 V2 [49] is an app designed for clinicians assessing ADHD. It is a semi-structured interview based on DSM-IV criteria for ADHD. Strenghts: based on DSM or ICD criteria for ADHD, easy to use, the results are easy to understand, limited time resources. Limitations: lack of information regarding the psychometric properties, no norms offered and in some cases interpretation is based on convenience samples (e.g., in line with other persons that have completed the test), limited assessment of functional impairments, several apps do not account for developmental changes in ADHD (e.g., the same items are used to assess ADHD in adults/parents as well as in children), some are not updated for DSM-5.

3.6. Empirical data described in the app content Very few apps (16 %) described the empirical support/ theory they draw on. In these ADHD apps, there were only references related to several professionals who had been involved in the development of the app, such as psychiatrists, doctors, neuroscientists, other professionals (e.g., police officer), or the description only mentioned that they were audited according to the WHO standards. Regarding the empirical evidence on the app efficacy, such data was not reported. For example, we found only two apps that were under current investigation; however, data was not available for any of the included ADHD apps.

3.7.2. Current empirical status of the treatment strategies used in the apps. Implications for clinicians In order to identify the empirical support we investigated whether the therapeutical strategies comprised in the treatment apps were supported by: (1) clinical guidelines, (2) meta-analyses and (3) randomized controlled trials (RCTs). Through this approach we can find out which apps are empirically supported, based on the existing research on the therapeutical package and which apps do not have a solid theoretical basis. Instead of conducting large RCTs to investigate the efficacy of an app in ameliorating ADHD symptoms, we can first investigate the strategies used in that app, select those apps that have the best empirical support for the strategies they use and conduct pilot studies and RCTs to investigate the effectiveness of those apps as compared to other treatments (e.g., treatment as usual, traditional face-to-face, or combined with medication or psychosocial treatments). In what follows we present the most promising apps judging by the empirical strategies they use.

3.7. Empirical status of the apps according to existing research Given the fact that conducting a randomized controlled trial involves significant costs, and that technology is evolving very fast with new apps being released every day and other apps disappearing from the mobile apps market [44], is is extremely important to investigate first the content of the apps in terms of the instruments used, symptoms and functional impairments assessed (for screening/assessment ADHD apps) and therapeutical strategies (for apps designed for ADHD treatment). 3.7.1. Current status of the screening/assessement ADHD apps In order to investigate the current status of the apps designed for ADHD screening and assessment, we analysed strenghts and limitations of the existing apps based on two main criteria: (1) whether the app-based assessment of ADHD is in line with the recommendations provided by clinical guidelines for ADHD assessment (e.g., NICE guidelines on diagnosis [6]). Taking this criterion into account, we tried to answer the next questions: (a) Does this app assess ADHD symptoms based on DSM or International Classification of Diseases (ICD) diagnostic criteria?

3.7.2.1. Pharmacological treatment monitoring apps. In this category, we identified several apps that allow physicians to collect data on symptoms and medication adherence from adult patients with ADHD or multiple informants (parents of children with ADHD, teachers). One app that is used with adults with ADHD is ADHD Health Storylines [50]. This is an app used by adults with ADHD to track their symptoms, their daily moods, their medication treatment. It allows the person to set 6

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

reminders (auditory and visual alerts) in order to adhere to the medication treatment. It also helps them to create routines and share the events saved in “My Storylines” section either with the care team or family/ friends. ADHD Psychopharmacology [51] is a very complex app that covers diagnosis as well as treatment; the app can be used by specialists. It can use input from parents and teachers as well. ADHD Doctors [52] is an app designed for professionals. It is designed for assessment (it is a scale, however, it seems to be based on a semistructured interview), and patient tracking (medication dose, treatment adherence, height/weight). The app was developed by the same company that made other two apps used for tracking and monitoring, one for children ADHD Kids [53] and one for adults ADHD Adults.

mixed regarding the impact of this form of treatment on ADHD symptoms, as there are differences according to the blind/not blind status of the rater. Results of a meta-analysis comparing neurofeedback with methylphenidate [63] indicated that the second intervention was more efficacious in reducing ADHD core symptoms, inattention and inhibition. Such an app is Super Powers for Super Kids [64]. - Cognitive training. This form of treatment was not included in clinical guidelines. Results of a meta-analysis [65] indicated that cognitive training had moderate effects on ADHD symptoms when outcomes were rated by not blinded raters (e.g., parents). Example of apps based on cognitive training are ADHD Trainer [66] and ADHD Adult Trainer [67]. - Hypnosis. Although we found several apps based on hypnosis, this treatment is not included in clinical guidelines (NICE guidelines [6]). We did not find empirical support coming from meta-analyses and there is a scarcity of studies investigating the efficacy of hypnotherapy for ADHD in youth or adults. We found only preliminary evidence for this form of treatment for adults with ADHD [68]. - Music therapy. Despite the fact that most treatment apps were based on music therapy, this treatment is not included in clinical guidelines (NICE guidelines [6]). We did not find meta-analyses, nor RCTs, we only found exploratory studies with few participants that were conducted in order to investigate the efficacy of music therapy for children diagnosed with ADHD [69].

3.7.2.2. Self-help and psychosocial treatment. In order to provide app users with a clear direction on the most useful apps, we analysed the content of the apps to see if they adhere to the clinical recommendations in guidelines for ADHD treatment. We also considered evidence coming from meta-analyses and randomized controlled trials for treatments/treatment strategies that were not included in the clinical guidelines. - Psychoeducation. Apps based on psychoeducation are in line with NICE guidelines [6], where parent training focused on ADHD is recommended as first line treatment for children under 5, while for children over the age of 5 and young people the provision of information regarding ADHD causes, impact, and parenting is recommended. From this category we recommend apps like: ADHD [54] and ADHD in Children [55], both developed by the same company. These apps provide information regarding the disorder, causes and available treatments for ADHD. The information is offered via animated videos but also a text transcript is available. ADHD Treatment [56] is another app covering information regarding causes and treatment for ADHD available on Android. On Itunes, ADHD Treatment [57] is an app covering information about ADHD symptoms, causes and treatment. This app is promising when it comes to information about ADHD symptoms and causes, but several treatments recommended in this app (treat ADHD with caffeine/ yoga) are not empirically supported. ADHD Angel [58] and ADHD Guardian Angel [59] are two apps developed by the same developer that can be used together. They provide psychoeducation about ADHD symptoms, the main difficulties encountered by people with ADHD (transition to high-school, problems with the law, difficulties with the peer group, school and job-related difficulties, driving a car or riding a bike), supply reports for medication, mood and symptom tracking. Finally, Jumpy car [60] is a gamified app about ADHD in which a child with ADHD talks to a doctor about ADHD symptoms, causes and treatment. - Organizational skills training. For those apps based on organizational skills training we found evidence coming from a meta-analysis [61] of randomized controlled trials. Examples of apps are: Visual Schedule [62], ADHD Notebook. - Neurofeedback. We did not find support coming from clinical guidelines for this form of treatment; however, several meta-analyses and randomized controlled trials have been conducted and the results are

3.8. Analysis of the top more downloaded ADHD apps Table 4 presents the characteristics of the most downloaded 4 apps on ADHD. Given that the number of downloads was available only in Google Play, we selected those apps that had the most downloads, which left us with a total of four. Out of these, three have been downloaded over 10,000 times and one app has been downloaded between 100,000 and 500,000 times. As we were interested in investigating the characteristics of the apps that were most downloaded, we conducted an analysis of their content and design. Three of the most downloaded ADHD apps were designed for assessment purposes only, one app was intended both for assessment and treatment. The ADHD Test has two options for the assessment of ADHD symptoms: a quick test and a complete test. The first version comprises 6 items, while the second version comprises 24 items, both scales are rated on a 6 point Likert scale. The app is very easy to use, as after completing the questionnaire it offers the user a raw score and an interpretation of that score. Also, it mentions very clearly that the app does not provide a clinical diagnosis, and it is only a self-report questionnaire, in consequence, a professional should be consulted. The name of the questionnaire, namely the Jasper/Goldberg Adult ADD Questionnaire, is also mentioned. The app is not interactive, it contains only written material on a white background, with no images or videos. For the second app, ADHD Adults, security and privacy are well established from the beginning. In order to have access to the app content the user must create a login account by entering his/her e-mail address and setting a password. Only after logging in, does the user have access to the app content. A complex app, it also provides the possibility to contact a doctor (via another related app, ADHD Doctors), is available only for those with a premium account or is free for 30 days

Table 4 Characteristics of the 4 most downloaded ADHD apps on Google Play. Name of the app

Producer

Costs

Purpose

Target population

Downloads

Ratings

Version

Last updated

ADHD Test ADHD Adults ADHD-Test Lite* ADHD Self Test

Individual Company Company Company

Free Free (in-app purchases) Free Free

A A&T A A

Adults Adults Adults and children Adults

100,000–500,000 10,000–50,000 10,000–50,000 10,000–50,000

3.3 3.9 3.9 3.9

Not reported 1.1.2 1.1 1.2

September 2012 January 2016 January 2014 May 2015

Note. ADHD = Attention-deficit/hyperactivity disorder, A = Assessment, T = Treatment. * app no longer available in March 2019. 7

(383 reviews) (71 reviews) (100 reviews) (47 reviews)

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

trial. The app allows ADHD assessment based on the Adult ADHD selfreport scale 1.1 containing 18 items, and it states that it is based on the DSM-IV criteria. After completing the assessment, no score (raw or standardized) is offered only an explanation on the presence of ADHD symptoms. The user is further invited to contact the doctor in case he/ she has difficulties understanding the explanation presented. The app can also be used for treatment, as it provides symptom management, enables notification setting, assessments planning, and scheduling meetings or tasks (for the current day, next three days, or the whole week). Also, users can introduce their medical history. The legal terms and privacy policy of the app are available. As for the design of the app, only written material is offered, with few images (for the main components of the app or charts about progress) and no videos. The app ADHD - Test Lite is designed for assessment purposes, and can be used both by adults and children. The app has two options from which the user can choose: a quick test (40 items) or an advanced test (> 350 items). The advanced test is available only for the pro version of the app, which is paid. The report that the app generates after completing the items offers a raw score and explains the presence of the symptoms based on the cut-off score of 50. Information about the predictive values of the test to identify ADHD is also offered (80 % for the quick version and 100 % for the advanced version). It also includes general information about the condition, such as prevalence and symptoms. The design of the app is straightforward, only written material is provided on a gray background. ADHD Self –Test is designed exclusively for assessment purposes. The app delves straight into the questions with no prior instructions for the user. The questionnaire consists of 24 items, rated on a 6-point Likert scale. The report issues a raw score and explains that scores above 70 are indicative of ADHD. Also, the app has several links to ADHD diet tips and music therapy as well as towards natural remedies. The app design is very simple and only written material is available. All four apps were offered free, with an updated version, simple design and information presented clearly in an accessible language. None of the apps provided personalization according to the user.

interventions delivered via apps for people with ADHD should take into consideration several aspects before being recommended in clinical practice. For instance, Hill and collaborators [75] proposed several recommendations to overcome potential challenges associated with leading-edge technologies for mental health problems, such as financial planning of the digital intervention, collaboration between specialists, commercial enterprises and clinical services, establishing the status of intellectual property beforehand, and finding proper statistic designs that allow to test the efficacy of digital interventions. It is no news that parents of ADHD children search for information over the Internet [76] on ADHD in general or regarding more specific aspects related to ADHD medication treatment. Although we identified over one hundred apps developed for ADHD, the level of evidence on which these apps are based as described in their content is lacking. These findings are in line with the results of a systematic review on the level of evidence for the existing technologies for children and adolescents with ADHD [77]. Powell and collaborators [77] also included in their systematic review studies in which interventions for youths with ADHD were delivered via mobile apps. Following systematic searches in major databases, they only identified two studies (a single case study and a case study) that were included in their systematic review. Negative effects of mental health interventions are important to consider, as research shows that deterioration occurs for several patients in both face-to-face and Internet-delivered therapy [78,79]. Moreover, there is limited research conducted on the negative effect of mental health interventions delivered via apps. Apps can have a detrimental effect as it is shown in one study in which an intervention for problem drinking delivered via an app was associated with opposite results, male participants’ problematic drinking consumption increased after receiving the intervention [80]. Users’ needs should be integrated into developing effective mental health apps, as was found in a qualitative study in which several focus groups were conducted with youths with ADHD and their parents as well as with professionals [81]. The results of two studies investigating the views of parents, children and specialists on several ADHD apps showed that the apps analyzed did not cover the needs of the sample investigated [27,28]. For instance, children with ADHD and clinicians working with them identified the following characteristics of the apps as being important: visual aspect of the app, tailoring, music content, audio feedback, immediate rewards, interactivity, monitoring and encouraging relationships with others. Nevertheless, the ADHD apps considered failed to meet these criteria. For parents, the most important characteristics were the visual aspect of the app, to be customizable, to help with the difficulties associated with the disorder, to provide the possibility to use the app together in order to improve the parent-child relationship, to be able to use the app as a source of information (for instance, by siblings), to help with monitoring, to be user-friendly and helpful with daily activities. The results showed again that the existing ADHD apps fail to meet the needs and expectations of parents related to content, aims and interface of ADHD apps. However, these results need to be interpreted carefully since they are based on the asnwers of a small sample of respondents: 7 parents, 6 clinicians working with parents, 5 children with ADHD and 5 clinicians working with ADHD children. ADHD apps need to be tested so as to prevent pseudoscientific marketing that promises individuals who use such apps a “remedy for all their ADHD-related problems”. Future studies should consider investigating the efficacy of both the therapeutic package and theory on which this package is based (see ref [82] for a review of psychological interventions for ADHD that simultaneously analyzed the existing evidence for the treatment efficacy and for the subsequent theory). Given the limited evidence that comes to support the effectiveness of mental health apps in general [83], the use of mental health apps in clinical practice should be done cautiously, by taking into consideration the potential risks for patients. Investigating the efficacy of ADHD apps in large randomized controlled trials, in comparison with active conditions (pharmacotherapy,

4. Discussion Even though evidence-based assessment and treatment exist for ADHD in youths and adults, several barriers are associated with access to them [70,71]. In fact, according to a recent study on ADHD treatment offered among US children and adolescents with ADHD, results showed that only a small percentage of children with ADHD receive psychosocial interventions [72]. The efficacy of digital health interventions for mental health problems is supported by randomized controlled trials and meta-analyses [13,73]. Currently, there are numerous available apps for mental health conditions [74]; however, there is a huge disproportion between the existing research on their efficacy and the number of the ADHD apps available in the commercial market. There are numerous ADHD apps available in app stores; nevertheless, research concerning their efficacy/ effectiveness is scarce. The aim of this paper is to provide a systematic review of the mental health apps for ADHD available in virtual stores and scientific databases. Our results indicate that numerous apps, 109 apps to be more precise, are available in the marketplace for a variety of categories of users, such as youths (children or adolescents), adults, parents, teachers or professionals. The results of systematic searches in the literature (PsycINFO, PubMed, Web of Science, Scopus and Cochrane database) point to one study that meets all our inclusion criteria. As for the content and characteristics of the most downloaded ADHD apps (only those that were downloaded over 10,000 times), we identified four that have a simple design, and limited interactivity; most of them have been developed for assessment purposes and have updated versions. Given the high proliferation of mental health apps, with new mental health apps being disseminated without any empirical tests, 8

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

psychotherapy or placebo controls) could shed more light on the clinical efficacy of ADHD apps and help disentangle the so-called “digital placebo effect” [84]. The current review has important clinical implications for the diagnosis and treatment of ADHD. As regards the ADHD diagnosis, this review identifies over twenty apps that can be used in the assessment of ADHD symptoms. Parents and teachers are those who usually rate the behavior of children with ADHD. An app that allows to rate symptoms at home or at school (or in other contexts where symptoms occur) is a more ecological indicator of a child’s behavior as compared to retrospective reports on such behaviors. However, research comparing ADHD assessment via apps with paper-pencil questionnaires is lacking, therefore, future research should demonstrate their accuracy in symptom identification. Furthermore, the assessment conducted through mobile apps could also be an alternative to weekly assessments of symptom improvement after treatment sessions. The assessment of ADHD symptoms is very important. Even though many apps were designed to assess ADHD symptomatology, none of the apps mentioned the psychometric properties of the questionnaires used. We found no study published that investigates the psychometric properties of instruments that are available on ADHD apps (e.g., measurement invariance). Despite the fact that there is preliminary evidence coming from a systematic review showing that data from self-reported assessments obtained from mobile apps is comparable to other modalities (e.g., paper, laptop, tablet, SMS) [85]), more research needs to be conducted on this subject. Establishing the psychometric properties is important for evidence-based assessments and future studies should investigate the equivalence of the instruments involved in the assessment of ADHD symptoms delivered via mental health apps with other methods to assess this condition that have already been validated. The relevance of this review for ADHD treatment lies in the fact that we found numerous apps that could be used in the treatment of ADHD in youths and adults. Such apps could be used effectively by therapists in their clinical practice. Let’s consider, for instance, the psychoeducational component of the apps. Therapists can recommend parents, teachers or patients to read information regarding ADHD symptoms and treatment from a free ADHD app. They can read such information at their own pace and come back to the content whenever they need. Given the widespread use of smartphones, it would be more practical for many users (caregivers, teachers, patients) to read such information on their smartphone. Given young people’s preference for technology, apps based on organizational skills training could also be a useful tool in treatment (e.g., for homework assessments). Future studies could inform us of the effectiveness of such tools used as enhancements to standard treatments as compared to standard treatments alone. Our study should be interpreted considering several limitations. Firstly, given the fact that the app market is expanding fast, changes in the number of ADHD available apps, as well as in user ratings, the number of downloads or other characteristics is inevitable to change over time. In fact, the virtual stores, as well as the NHS Apps Library, were checked at two later dates (March and September 2019) and some changes are reported, as for instance, one of the most downloaded apps in Google Play was no longer available (12/03/2019). Second, another significant limitation of this paper is the fact that we confined our search to the two most popular stores (Apple App Store and Google Play) and the NHS Apps Library database, and we did not conduct systematic searches on other stores or operating systems. Other ADHD apps might exist but have not been included in this review. Furthermore, the two searches, one in the virtual stores (May 2017) and one in the literature (September 2019) were conducted at different times. Even so, we were not able to find a close correspondence between mobile apps available in the virtual stores and in the literature. The single eligible study identified in the literature search, investigating the efficacy of a mobile app in improving medication adherence and ADHD severity [39], was published outside of the 2017 virtual store search, so there is no overlap between evidence in the literature and

apps on the market. Even though the authors of this study provided links for the app stores where the app can be installed, the links were not functional. We did not include this app in our systematic review, as at the time of our initial search in the virtual stores, as well as on the two later checks from March and September 2019, we have not able to find the app in Apple App Store, nor in Google Play. Therefore, with few exceptions, where we were able to find in the literature either conference papers on app development, or case reports for apps identified in the virtual stores [30,32], there is a huge gap between commercially available mobile apps for ADHD and research on evidence-based interventions for ADHD published in the literature. We did not find any eligible ADHD apps by searching in the NHS Apps Library, and one potential explanation for this could be related to their policy/ criteria established for the acceptance of apps. Therefore, in order to be included in the NHS Apps Library, apps need to be either tested or approved by the National Health Service, and research on their efficacy on outcomes, user experience, safety, harm and user experience with the app needs to be available. A first step, for instance, is to pass a Digital Assessment Questionnaire, where along with information about the digital tool, information on the evidence on which the app was designed (is the content in accordance with guidelines, research or clinical opinions?), existing or ongoing research regarding the app must be presented. As our review indicates, this information is missing for most of the apps. Also, information on privacy, security, data storage and cryptography are required to register the app, and according to this review, only a few ADHD apps have a stated privacy policy accessible to the user. 5. Conclusions The aim of this paper was to conduct a systematic review of the ADHD apps in two of the most used stores (Apple App Store and Google Play), the NHS Apps Library and the literature, to describe their characteristics, present their empirical status, as well as describe the content and design of four of the most downloaded apps. Our findings show that numerous mental health apps are available for those suffering from ADHD and their caregivers. More precisely, we identified 109 apps in virtual stores and one additional app in our systematic search in the literature that corresponded to our inclusion criteria. However, such treatments need to be investigated rigorously before being advanced to patients with ADHD. There needs to be collaborative work between specialists and companies/individuals who develop mental health apps, and such apps should not be available, either for free or at any cost, before being first tested in randomized controlled trials. Summary points is a prevalent condition associated with negative con• ADHD sequences for individuals, families, and society. treatments for ADHD exist, but barriers in • Evidence-based accessing them are still a major problem. apps are promising treatment paths in many mental • Mobile health conditions. first systematic review that explores apps available in • The IOS/ Google Play, the NHS Apps Library and the literature. apps covering assessment, treatment and mixed assess• 110 ment and treatment for ADHD, delivered for youth, adults, professionals.

Author contributions Study conception and design: CRP Acquisition of data: CRP Analysis and interpretation of data: CRP, AD, GA 9

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al.

Drafting of manuscript: CRP Critical revision: GA, AD All authors read and approved the final manuscript.

(2015) 24–31. [17] N. Wrońska, B. Garcia-Zapirain, A. Mendez-Zorrilla, An iPad-Based tool for improving the skills of children with attention deficit disorder, Int. J. Environ. Res. Public Health 12 (2015) 6261–6280, https://doi.org/10.3390/ijerph120606261. [18] N. Franke, L.J. Keown, M.R. Sanders, An RCT of an online parenting program for parents of preschool-aged children with ADHD symptoms, J. Atten. Disord. (2016), https://doi.org/10.1177/1087054716667598. [19] A.C. Powell, M. Chen, C. Thammachart, The economic benefits of mobile apps for mental health and telepsychiatry services when used by adolescents, Child Adolesc. Psychiatr. Clin. N. Am. 26 (2017) 125–133, https://doi.org/10.1016/j.chc.2016.07. 013. [20] R. Grist, J. Porter, P. Stallard, Mental health mobile apps for preadolescents and adolescents: a systematic review, J. Med. Internet Res. 19 (2017) e176, https://doi. org/10.2196/jmir.7332. [21] J. Nicholas, A.S. Fogarty, K. Boydell, H. Christensen, The reviews are in: a qualitative content analysis of consumer perspectives on apps for bipolar disorder, J. Med. Internet Res. 19 (2017) e105, https://doi.org/10.2196/jmir.7273. [22] J. Nicholas, M.E. Larsen, J. Proudfoot, H. Christensen, Mobile apps for bipolar disorder: a systematic review of features and content quality, J. Med. Internet Res. 17 (2015) e198, https://doi.org/10.2196/jmir.4581. [23] M. Van Ameringen, J. Turna, Z. Khalesi, K. Pullia, B. Patterson, There is an app for that! The current state of mobile applications (apps) for DSM-5 obsessive-compulsive disorder, posttraumatic stress disorder, anxiety and mood disorders, Depress. Anxiety (2017), https://doi.org/10.1002/da.22657. [24] M. Alyami, B. Giri, H. Alyami, F. Sundram, Social anxiety apps: a systematic review and assessment of app descriptors across mobile store platforms, Evid. Based Ment. Health 20 (2017) 65–70, https://doi.org/10.1136/eb-2017-102664. [25] A. Huguet, S. Rao, P.J. McGrath, L. Wozney, M. Wheaton, J. Conrod, S. Rozario, A systematic review of cognitive behavioral therapy and behavioral activation apps for depression, PLoS One 11 (2016) e0154248, , https://doi.org/10.1371/journal. pone.0154248. [26] M.E. Larsen, J. Nicholas, H. Christensen, A systematic assessment of smartphone tools for suicide prevention, PLoS One 11 (2016) e0152285, , https://doi.org/10. 1371/journal.pone.0152285. [27] L. Powell, J. Parker, V. Harpin, ADHD: Is There an App for That? A Suitability Assessment of Apps for the Parents of Children and Young People With ADHD, JMIR Mhealth Uhealth 5 (2017) e149, https://doi.org/10.2196/mhealth.7941. [28] L. Powell, J. Parker, N. Robertson, V. Harpin, Attention deficit hyperactivity disorder: is there an app for that? Suitability assessment of apps for children and young people with ADHD, JMIR Mhealth Uhealth 5 (2017) e145, https://doi.org/10. 2196/mhealth.7371. [29] Z. Young, M.P. Craven, M. Groom, J. Crowe, Snappy app: a mobile continuous performance test with physical activity measurement for assessing attention deficit hyperactivity disorder, Human-Computer Interaction. Applications and Services, Springer, Cham, 2014, pp. 363–373, https://doi.org/10.1007/978-3-319-072272_35. [30] D. Spachos, A. Chifari, G. Chiazzese, G. Merlo, G. Doherty, P. Bamidis, WHAAM: a mobile application for ubiquitous monitoring of ADHD behaviors, 2014 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL2014) (2014) 305–309, https://doi.org/10.1109/IMCTL.2014. 7011153. [31] S. Schuck, N. Emmerson, H. Ziv, P. Collins, S. Arastoo, M. Warschauer, F. Crinella, K. Lakes, Designing an iPad app to monitor and improve classroom behavior for children with ADHD: iSelfControl feasibility and pilot studies, PLoS One 11 (2016) e0164229, , https://doi.org/10.1371/journal.pone.0164229. [32] G. Ruiz-Manrique, K. Tajima-Pozo, F. Montañes-Rada, Case Report: “ADHD Trainer”: the mobile application that enhances cognitive skills in ADHD patients, F1000Res 3 (2015), https://doi.org/10.12688/f1000research.5689.3. [33] J.E. Pinna, Development and Preliminary Testing of Tablet Application to Increase Reading Motivation and Summarization for Adolescent Students with ADHD, University of Baltimore, 2015. [34] E. Schoenfelder, M. Moreno, M. Wilner, K.B. Whitlock, J.A. Mendoza, Piloting a mobile health intervention to increase physical activity for adolescents with ADHD, Prev. Med. Rep. 6 (2017) 210–213, https://doi.org/10.1016/j.pmedr.2017.03.003. [35] D. Moher, A. Liberati, J. Tetzlaff, D.G. Altman, T.P. Group, Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement, PLoS Med. 6 (2009) e1000097, , https://doi.org/10.1371/journal.pmed.1000097. [36] I. SPSS, IBM SPSS Statistics for Windows, Version 20.0, IBM Corp, New York, 2011. [37] J. Cohen, A coefficient of agreement for nominal scales, Educ. Psychol. Meas. 20 (1960) 37–46, https://doi.org/10.1177/001316446002000104. [38] J.R. Landis, G.G. Koch, The measurement of observer agreement for categorical data, Biometrics 33 (1977) 159–174. [39] O. Weisman, Y. Schonherz, T. Harel, M. Efron, M. Elazar, D. Gothelf, Testing the efficacy of a smartphone application in improving medication adherence, among children with ADHD, Isr. J. Psychiatry Relat. Sci. 55 (2018) 59–63. [40] J.M. Swanson, H.C. Kraemer, S.P. Hinshaw, L.E. Arnold, C.K. Conners, H.B. Abikoff, W. Clevenger, M. Davies, G.R. Elliott, L.L. Greenhill, L. Hechtman, B. Hoza, P.S. Jensen, J.S. March, J.H. Newcorn, E.B. Owens, W.E. Pelham, E. Schiller, J.B. Severe, S. Simpson, B. Vitiello, K. Wells, T. Wigal, M. Wu, Clinical relevance of the primary findings of the MTA: success rates based on severity of ADHD and ODD symptoms at the end of treatment, J. Am. Acad. Child Adolesc. Psychiatry 40 (2001) 168–179, https://doi.org/10.1097/00004583-200102000-00011. [41] R.C. Kessler, L. Adler, M. Ames, O. Demler, S. Faraone, E. Hiripi, M.J. Howes, R. Jin, K. Secnik, T. Spencer, T.B. Ustun, E.E. Walters, The World Health Organization Adult ADHD Self-Report Scale (ASRS): a short screening scale for use in the general population, Psychol. Med. 35 (2005) 245–256.

Declaration of Competing Interest None. Funding This work was supported by a grant from the Romanian Executive Unit for Financing Higher Education Research, Development and Innovation [grant number PN-III-P4-ID-PCE-2016-0861]. References [1] E.S. Vitola, C.H.D. Bau, G.A. Salum, B.L. Horta, L. Quevedo, F.C. Barros, R.T. Pinheiro, C. Kieling, L.A. Rohde, E.H. Grevet, Exploring DSM-5 ADHD criteria beyond young adulthood: phenomenology, psychometric properties and prevalence in a large three-decade birth cohort, Psychol. Med. 47 (2017) 744–754, https://doi. org/10.1017/S0033291716002853. [2] R. Thomas, S. Sanders, J. Doust, E. Beller, P. Glasziou, Prevalence of AttentionDeficit/Hyperactivity disorder: a systematic review and meta-analysis, Pediatrics 135 (2015) e994–e1001, https://doi.org/10.1542/peds.2014-3482. [3] M. Fridman, T. Banaschewski, V. Sikirica, J. Quintero, M.H. Erder, K.S. Chen, Factors associated with caregiver burden among pharmacotherapy-treated children/adolescents with ADHD in the Caregiver Perspective on Pediatric ADHD survey in Europe, Neuropsychiatr. Dis. Treat. 13 (2017) 373–386, https://doi.org/ 10.2147/NDT.S121391. [4] K. Gupte-Singh, R.R. Singh, K.A. Lawson, Economic burden of attention-deficit/ hyperactivity disorder among pediatric patients in the United States, Value Health 20 (2017) 602–609, https://doi.org/10.1016/j.jval.2017.01.007. [5] American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders (DSM-5®), American Psychiatric Association, Arlington, VA, 2013. [6] National Institute for Health and Care Excellence. Attention deficit hyperactivity disorder: diagnosis and management. https://www.nice.org.uk/guidance/ng87 (Accessed 13 March 2019). [7] E.J.S. Sonuga-Barke, D. Brandeis, S. Cortese, D. Daley, M. Ferrin, M. Holtmann, J. Stevenson, M. Danckaerts, S. van der Oord, M. Döpfner, R.W. Dittmann, E. Simonoff, A. Zuddas, T. Banaschewski, J. Buitelaar, D. Coghill, C. Hollis, E. Konofal, M. Lecendreux, I.C.K. Wong, J. Sergeant, European ADHD Guidelines Group, Nonpharmacological interventions for ADHD: systematic review and metaanalyses of randomized controlled trials of dietary and psychological treatments, Am. J. Psychiatry 170 (2013) 275–289, https://doi.org/10.1176/appi.ajp.2012. 12070991. [8] S. Cortese, N. Adamo, C. Del Giovane, C. Mohr-Jensen, A.J. Hayes, S. Carucci, L.Z. Atkinson, L. Tessari, T. Banaschewski, D. Coghill, C. Hollis, E. Simonoff, A. Zuddas, C. Barbui, M. Purgato, H.-C. Steinhausen, F. Shokraneh, J. Xia, A. Cipriani, Comparative efficacy and tolerability of medications for attentiondeficit hyperactivity disorder in children, adolescents, and adults: a systematic review and network meta-analysis, Lancet Psychiatry 5 (2018) 727–738, https://doi. org/10.1016/S2215-0366(18)30269-4. [9] S.-H. Shim, H. Yoon, J. Bak, S.-W. Hahn, Y.-K. Kim, Clinical and neurobiological factors in the management of treatment refractory attention-deficit hyperactivity disorder, Prog. Neuropsychopharmacol. Biol. Psychiatry 70 (2016) 237–244, https://doi.org/10.1016/j.pnpbp.2016.04.007. [10] G. Andersson, N. Titov, Advantages and limitations of Internet-based interventions for common mental disorders, World Psychiatry 13 (2014) 4–11, https://doi.org/ 10.1002/wps.20083. [11] D.C. Mohr, M.N. Burns, S.M. Schueller, G. Clarke, M. Klinkman, Behavioral Intervention Technologies: evidence review and recommendations for future research in mental health, Gen. Hosp. Psychiatry 35 (2013) 332–338, https://doi.org/ 10.1016/j.genhosppsych.2013.03.008. [12] E. Murray, E.B. Hekler, G. Andersson, L.M. Collins, A. Doherty, C. Hollis, D.E. Rivera, R. West, J.C. Wyatt, Evaluating digital health interventions, Am. J. Prev. Med. 51 (2016) 843–851, https://doi.org/10.1016/j.amepre.2016.06.008. [13] C.R. Păsărelu, G. Andersson, L. Bergman Nordgren, A. Dobrean, Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials, Cogn. Behav. Ther. 46 (2017) 1–28, https://doi.org/10.1080/16506073.2016.1231219. [14] C. Hollis, C.J. Falconer, J.L. Martin, C. Whittington, S. Stockton, C. Glazebrook, E.B. Davies, Annual Research Review: digital health interventions for children and young people with mental health problems - a systematic and meta-review, J. Child Psychol. Psychiatry 58 (2017) 474–503, https://doi.org/10.1111/jcpp.12663. [15] R. Pettersson, S. Söderström, K. Edlund-Söderström, K.W. Nilsson, Internet-based cognitive behavioral therapy for adults with ADHD in outpatient psychiatric care: a randomized trial, J. Atten. Disord. 21 (2017) 508–521, https://doi.org/10.1177/ 1087054714539998. [16] B. Moëll, L. Kollberg, B. Nasri, N. Lindefors, V. Kaldo, Living SMART—a randomized controlled trial of a guided online course teaching adults with ADHD or subclinical ADHD to use smartphones to structure their everyday life, Internet Interv. 2

10

International Journal of Medical Informatics 138 (2020) 104133

C.R. Păsărelu, et al. [42] J.J.S. Kooij, Adult ADHD: Diagnostic Assessment and Treatment, Springer Science & Business Media, 2012. [43] HealthyVisions, ADHD Concentration: Noble Isle, HealthyVisions, 2017, https:// play.google.com/store/apps/details?id=com.healthyvisions.nobleisle. [44] S.R. Chan, S. Misra, Certification of mobile apps for health care, JAMA 312 (2014) 1155–1156, https://doi.org/10.1001/jama.2014.9002. [45] ADHD Self Test – Google Play, (n.d.). https://play.google.com/store/apps/details? id=com.soundaddadhdtreatment&hl=ro (Accessed 16 July 2019). [46] ADHD Test, App Store. (n.d.). https://apps.apple.com/us/app/adhd-test/ id690050861 (Accessed 16 July 2019). [47] Attention Deficit ADHD Test, App Store. (n.d.). https://apps.apple.com/us/app/ attention-deficit-adhd-test/id395039893 (Accessed 16 July 2019). [48] ADHD Medtest, App Store. (n.d.). https://apps.apple.com/us/app/adhd-medtest/ id1092899357 (Accessed 16 July 2019). [49] Diva 2.0 V2 – Google Play, (n.d.). https://play.google.com/store/apps/details?id= de.gimik.apps.diva2&hl=ro (Accessed 16 July 2019). [50] ADHD Health Storylines, App Store. (n.d.). https://apps.apple.com/us/app/adhdhealth-storylines/id982558011 (Accessed 16 July 2019). [51] ADHD Psychopharmacology, App Store. (n.d.). https://apps.apple.com/us/app/ adhd-psychopharmacology/id353971353 (Accessed 16 July 2019). [52] ADHD Doctors - Google Play, (n.d.). https://play.google.com/store/apps/details? id=com.labshealth.tdahdoctors&hl=ko (Accessed 16 July 2019). [53] ADHD Kids – Google Play, (n.d.). https://play.google.com/store/apps/details?id= com.labshealth.tdahkids&hl=ro (Accessed 16 July 2019). [54] ADHD – Google Play, (n.d.). https://play.google.com/store/apps/details?id=com. focusmedica.ud.adhd&hl=ro (Accessed 16 July 2019). [55] FAQs - ADHD in Children – Google Play, (n.d.). https://play.google.com/store/ apps/details?id=com.focusmedica.ccfaqadhdinchildren&hl=ro (Accessed 16 July 2019). [56] ADHD Treatment – Google Play, (n.d.). https://play.google.com/store/apps/ details?id=com.adhdtreatment&hl=ro (Accessed 16 July 2019). [57] ADHD Treatment - Learn More About ADHD, App Store. (n.d.). https://apps.apple. com/us/app/adhd-treatment-learn-more-about-adhd/id1172819503 (Accessed 16 July 2019). [58] ADHD Angel, App Store. (n.d.). https://apps.apple.com/pa/app/adhd-angel/ id485821457?l=en (Accessed 16 July 2019). [59] ADHD Guardian Angel, App Store. (n.d.). https://apps.apple.com/pa/app/adhdguardian-angel/id635654273?l=en (Accessed 16 July 2019). [60] Jumpy Car ADHD - Free – Google Play, (n.d.). https://play.google.com/store/apps/ details?id=pt.ivopintoo.jumpy.car.adhd.free&hl=ro (Accessed 16 July 2019). [61] A. Bikic, B. Reichow, S.A. McCauley, K. Ibrahim, D.G. Sukhodolsky, Meta-analysis of organizational skills interventions for children and adolescents with AttentionDeficit/Hyperactivity Disorder, Clin. Psychol. Rev. 52 (2017) 108–123, https://doi. org/10.1016/j.cpr.2016.12.004. [62] Visual Schedule – Google Play, (n.d.). https://play.google.com/store/apps/details? id=com.intoronto.myschedule&hl=ro (Accessed 16 July 2019). [63] L. Yan, S. Wang, Y. Yuan, J. Zhang, Effects of neurofeedback versus methylphenidate for the treatment of ADHD: systematic review and meta-analysis of head-tohead trials, Evid. Based Ment. Health (2019), https://doi.org/10.1136/ebmental2019-300088 ebmental-2019-300088. [64] Super Powers for Super Kids, App Store. (n.d.). https://apps.apple.com/au/app/ super-powers-for-super-kids/id860725094 (Accessed 16 July 2019). [65] S. Cortese, M. Ferrin, D. Brandeis, J. Buitelaar, D. Daley, R.W. Dittmann, M. Holtmann, P. Santosh, J. Stevenson, A. Stringaris, A. Zuddas, E.J.S. SonugaBarke, European ADHD Guidelines Group (EAGG), Cognitive training for attentiondeficit/hyperactivity disorder: meta-analysis of clinical and neuropsychological outcomes from randomized controlled trials, J. Am. Acad. Child Adolesc. Psychiatry 54 (2015) 164–174, https://doi.org/10.1016/j.jaac.2014.12.010. [66] ADHD Trainer, App Store. (n.d.). https://apps.apple.com/us/app/adhd-trainer/ id687869470 (Accessed 16 July 2019). [67] ADHD Adult Trainer, App Store. (n.d.). https://apps.apple.com/us/app/adhdadult-trainer/id732776694 (Accessed 16 July 2019). [68] M. Virta, S. Hiltunen, M. Mattsson, S. Kallio, The impact of hypnotic suggestions on

[69] [70]

[71] [72]

[73]

[74] [75]

[76] [77]

[78] [79]

[80]

[81]

[82] [83] [84] [85]

11

reaction times in continuous performance test in adults with ADHD and healthy controls, PLoS One 10 (2015). F.F. Cripe, Rock music as therapy for children with attention deficit disorder: an exploratory study, J. Music Ther. 23 (1986) 30–37, https://doi.org/10.1093/jmt/ 23.1.30. N. Wright, M. Moldavsky, J. Schneider, I. Chakrabarti, J. Coates, D. Daley, P. Kochhar, J. Mills, W. Sorour, K. Sayal, Practitioner Review: Pathways to care for ADHD – a systematic review of barriers and facilitators, J. Child Psychol. Psychiatry 56 (2015) 598–617, https://doi.org/10.1111/jcpp.12398. P. Corkum, M. Bessey, M. McGonnell, A. Dorbeck, Barriers to evidence-based treatment for children with attention-deficit/hyperactivity disorder, Atten. Defic. Hyperact. Disord. 7 (2015) 49–74, https://doi.org/10.1007/s12402-014-0152-z. M.L. Danielson, S.N. Visser, A. Chronis-Tuscano, G.J. DuPaul, A national description of treatment among United States children and adolescents with attention-deficit/ hyperactivity disorder, J. Pediatr. (2017), https://doi.org/10.1016/j.jpeds.2017. 08.040. T.M. Fleming, L. Bavin, K. Stasiak, E. Hermansson-Webb, S.N. Merry, C. Cheek, M. Lucassen, H.M. Lau, B. Pollmuller, S. Hetrick, Serious games and gamification for mental health: current status and promising directions, Front. Psychiatry 7 (2017), https://doi.org/10.3389/fpsyt.2016.00215. A.L. Rathbone, J. Prescott, The use of mobile apps and SMS messaging as physical and mental health interventions: systematic review, J. Med. Internet Res. 19 (2017), https://doi.org/10.2196/jmir.7740. C. Hill, J.L. Martin, S. Thomson, N. Scott-Ram, H. Penfold, C. Creswell, Navigating the challenges of digital health innovation: considerations and solutions in developing online and smartphone-application-based interventions for mental health disorders, Br. J. Psychiatry (2017), https://doi.org/10.1192/bjp.bp.115.180372. A. Sage, D. Carpenter, R. Sayner, K. Thomas, L. Mann, S. Sulzer, A. Sandler, B. Sleath, Online information-seeking behaviors of parents of children with ADHD, Clin. Pediatr. (2017), https://doi.org/10.1177/0009922817691821. L. Powell, J. Parker, V. Harpin, What is the level of evidence for the use of currently available technologies in facilitating the self-management of difficulties associated with ADHD in children and young people? A systematic review, Eur. Child Adolesc. Psychiatry (2017) 1–22, https://doi.org/10.1007/s00787-017-1092-x. D.H. Barlow, Negative effects from psychological treatments: a perspective, Am. Psychol. 65 (2010) 13–20, https://doi.org/10.1037/a0015643. A. Rozental, K. Magnusson, J. Boettcher, G. Andersson, P. Carlbring, For better or worse: an individual patient data meta-analysis of deterioration among participants receiving Internet-based cognitive behavior therapy, J. Consult. Clin. Psychol. 85 (2017) 160–177, https://doi.org/10.1037/ccp0000158. M. Gajecki, A.H. Berman, K. Sinadinovic, I. Rosendahl, C. Andersson, Mobile phone brief intervention applications for risky alcohol use among university students: a randomized controlled study, Addict. Sci. Clin. Pract. 9 (2014) 11, https://doi.org/ 10.1186/1940-0640-9-11. L. Simons, A.Z. Valentine, C.J. Falconer, M. Groom, D. Daley, M.P. Craven, Z. Young, C. Hall, C. Hollis, Developing mHealth remote monitoring technology for attention deficit hyperactivity disorder: a qualitative study eliciting user priorities and needs, JMIR Mhealth Uhealth 4 (2016), https://doi.org/10.2196/mhealth. 5009. A. Dobrean, C.R. Păsărelu, M. Döpfner, D. David, S. Jay Lynn, G.H. Montgomery, Varieties of psychotherapy for attention-deficit hyperactivity disorder, Evid. Based Psychother. (2018) 435–522. J.H. Lui, D.K. Marcus, C.T. Barry, Evidence-based apps? A review of mental health mobile applications in a psychotherapy context, Prof. Psychol. Res. Pr. 48 (2017) 199. J. Torous, J. Firth, The digital placebo effect: mobile mental health meets clinical psychiatry, Lancet Psychiatry 3 (2016) 100–102, https://doi.org/10.1016/S22150366(15)00565-9. J.S. Marcano Belisario, J. Jamsek, K. Huckvale, J. O’Donoghue, C.P. Morrison, J. Car, Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods, Cochrane Database Syst. Rev. (2015) MR000042, , https://doi.org/10.1002/14651858.MR000042.pub2.