The below evaluation is an example of the types of information you may find when using our Model to assess specific apps. This sample evaluation is not intended to enable you to make a decision about choosing these specific apps for actual use, as such a decision is a personal one that you will make based on your own unique preferences, needs, and evaluation. We strongly encourage you to download these apps for yourself and use our Model to conduct your own assessment.
This evaluation was completed May 2020, and is based on publicly available information found by our Panel at that time. In July 2020, APA's contacted the app developers and offered to them an opportunity to provide feedback on these assessments. Comments from developers who responded are included here. These comments do not necessarily reflect publicy available information at the time of the original assessment or presently.
Access & Background
On which platforms/operating systems does the app work? Does it also work on desktop?
IOS, Android and Computer.
From SuperBetter: There is an iOS app, an Android app and a website version.
Has the app been updated in the last 180 days?
1 year ago V1.19
Does the app identify ownership?
Yes: SuperBetter LLC.
Does the app identify funding sources and conflicts of interest?
Funding sources and conflicts of interest are unclear to the reviewers based on information from the developer website or the app itself.
From SuperBetter: The company's only external funding to date is $170k from private investors. Funding sources can be found in the Crunchbase link cited as a source in the answer about "trusted source.
Does the app come from a trusted source?
Evidence for users to make this decision: CEO - Keith Wakeman and Chief Science Officer Jane McGonigal (she is author of a NYT bestselling book "Super Better").
From SuperBetter: Yes, evidence for users to make this decision: SuperBetter was invented by Jane McGonigal PhD. Jane is a researcher, world-renowned game designer and the company's Chief Science Officer. She co-founded the SuperBetter company along with CEO Keith Wakeman. The app has been evaluated in published studies conducted at UPENN and The Ohio State University Wexner Medical Center, and has been played by over 1 million people. https://www.superbetter.com/about
Does the app claim to be medical?
No: User terms indicate they are not responsible for anything that happens (adverse) because they are not complient with FDA because they are not a medical app.
From SuperBetter: No, the app does not claim to be medical. SuperBetter also comments that, as a non-medical app, it is "not subject to FDA regulation as a medical advice, so it is not out of compliance for what it is. SuperBetter notes that the reviewer response about "liability for adverse events also does not directly address the question," and provides input below.
Are there additional or hidden costs?
Developer states that the app is free in the FAQ, but in the Terms and Conditions states there is a subscription plan.
From SuperBetter: SuperBetter is a free app. There are no hidden costs. The Terms & Conditions includes a section describin terms "IF SuperBetter adds paid services in the future and you choose to purchase a paid service."
Does the app work offline?
It does not appear to be. This was tested by reviewers using the app in Airplane Mode.
It does not appear to be. This was tested by reviewers using the app in Airplane Mode.
Does the app work with accessibility features of the iPhone/Android? Is it accessible for those with impaired vision or other disabilities?
These reviewers found that the app is partially responsive to accessibility features. Not able to turn on captions, can turn on and off sound. Difficulties with creating larger font and unable to modify colors for color blindness. The app also works with screen reader.
Privacy & Safety
Does the app collect, use, and/or transmit sensitive data? If yes, does it claim to do so securely?
Yes, there is a potential to transmit sensitive data through the chat/communication forum. This is done at the user's discretion.
Does the app declare data use and purpose?
Yes, app discusses how data is obtained, stored and used.
Does the app describe use of PHI?
Yes, app describes use of non-personal information and personally identified information.
Can you opt out of data collection or delete data?
App terms mention that you can opt out of cookies, but may miss out on certain features. There is an option to mail a request to SuperBetter for the following: "Upon your request, we will delete your Personally Identifiable Information or Personal Health Information from our active databases and where feasible from our back-up media. You should be aware that it is not technologically possible to remove each and every record of the information you have provided to SuperBetter from our servers."
Is data maintained in the device or on the web?
Data is maintained on servers.
Does the app explain security systems used?
Yes, the terms mention some of the security features such as password protection, firewall and encryption.
What third parties does the app share data with?
Third parties are not specifically listed but the app utilizes multiple third parties. There is an option to sign up through Facebook and in the activity post, option to interface with Twitter.
From SuperBetter: SuperBetter offers clarification: Third parties are not specifically listed, but the app utilizes multiple third parties. There is an option to sign up and login through Facebook. At their discretion, users are able to share activities they complete in the app as posts on Twitter and Facebook.
If appropriate, is the app equipped to respond to potential harms or safety concerns?
No, terms state that app is not responsible for harm or death.
From SuperBetter: No, the app is not equipped to respond to potential harms or safety concerns. The terms & conditions direct the user to contact their doctor in the event of a medical emergency. SuperBetter notes that the intent of this question is about capabilities,not liability copy in the terms & conditions.
Does the app appear to do what it claims to do?
Yes, in theory. It appears to use gamification to encourage healthy habits.
From SuperBetter: Yes, the app claims to build resilience and improve mental health by bringing the psychology and mindset of game play to real life. Its resilence-building benefits and links to published studies evaluating the app are found on the Science page of the website. SuperBetter notes that they use the term "Gameful Design" instead of gamification.
Does the app have a clinical/recovery foundation relevant to your intended use?
According to these reviewers, yes. App does not indicate specificially for children under the age of 13. Screen reading is available so child wouldn't necessarily need to know how to read.
From SuperBetter: The studies cited above that evaluated the app provide evidence of a clinical foundation. Note that the requirement that individuals must be at least 13 to create an account is to protect the privacy of chidren.
Is the app content correct, well-written, and relevant?
According to these reviewers, yes. Each section states the evidence exists to support the activity (though no citiations are given).
From SuperBetter: SuperBetter adds more detail to the reviewers' response: Each section states the evidence exists to support the activity. There are citations in the Science cards found in the app. Also, the SuperBetter book describes the app's methodology and includes nearly 300 citations.
What are the relevant sources or references supporting the app use cases?
App links journal articles written by company executives, as well as journal articles that discuss health benefits of mobile apps in general.
From SuperBetter: The app has been evaluated in two published studies. In a randomized controlled trial at the Univeristy of Pennsylvania, SuperBetter players had significanlty greater reduction in symptoms of depression and anxiety compared to a control group. In an NIH-funded clinical trial at The Ohio State University Wexner Medical Center and Cincinnati Children's Hospital, teen patients with persistent concussion who played SuperBetter in conjunction with medical care had greater reductions in symptoms and increases in optimism compared to a control group who only received medical care. In published meta-analyses of randomized controlled trials evaluating smartphone apps SuperBetter had the greatest effect size for reducing symptoms of anxiety and depression.
Was there an attempt to validate app usability and feasibility?
Yes, there are two documented studies where they attempt to validate the app to show benefit in reduction of depressive symptoms in adults or reducing symptoms of concussions in teens.
From SuperBetter: Yes, the published studies mentioned above validated usability and feasibility. Over 1 million people have played SuperBetter, which is addtiional evidence of its usability and feasability.
Does the app seem easy to use?
According to these reviewers, the app is easy to use and visually appealing.
What are the main engagement styles of the app?
Gamification, Facebook group, notifications, text based questions.
From SuperBetter: SuperBetter is experienced as a game played in real life. The main engagement is daily activities in the app that prompt actions in the real world. Users earn resilience points for each completed activity. The app has push notifications. There is a SuperBetter Community Group on Facebook. SuperBetter notes that the phrase 'Text based questions' is confusing, it implies that questions are sent by text which is not accurate.
Do the app and its features align with your needs and priorities?
Yes for needs of building resilince for children, young adults and those that prefer a gaming like experience.
From SuperBetter: SuperBetter would remove the world "children" and replace it would "youth."
Is it customizable?
Yes, users can select the types of quesitons, issues, etc, to focus on such as depression, anxiety, other concerns.
From SuperBetter: Yes, users can select their challenge area and content (Power Packs). They set their own goals (epic wins), and can create and customize activities and choose their own allies to recruit for support.
Does the app clearly define its functional scope?
Yes, the app promotes itself as a tool using gamification and other features to address users' health issues/concerns.
From SuperBetter: Yes, the app promotes itself as a tool using gameful design and other features to build reselience and address users' health issues/concerns.
Can data be easily shared and interpreted in a way that's consistent with the stated purpose of the app?
Unclear to these reviewers. It seems like data can be shared by the user through Facebook, Twitter or activity forum. The option to export data for email or otherwise was not readily observable through the app. It is possible to share data with third party apps inadvertantly by clicking on ads or other third party links.
From SuperBetter: No, data cannot easliy be shared and interpreted by providers. The web version of the user has a Secret Lab area that includes data tracking but it is for the user only and not designed to be shared. SuperBetter notes that there are no ads in the app, so this should not be a concern to the user.
Do you own your data?
From what these reviewers could find, this is not explicity stated in the terms and or policy.
Can the app share data with EMR and other data tools (Apple HealthKit, FitBit)?
The app does not appear to interface with EMR or other health data tools based on these reviewers' examination.
Is the app for individual use or to be used in collaboration with a provider? If intended to be used with a provider, does the app have the ability to export or transfer data?
A provider could have the app and be an ally through the app, but the app does not explicitly encourage this. Additional functionality to export data was not readily apparent.
Does the app lead to any positive behavior change or skill acquisition?
It purports to impact and improve relisience, mood, working with others and gain skills in those areas as well.
Does the app improve therapeutic alliance between patient and provider?
Not a stated purpose but possibly could, according to these reviewers.