BACKGROUND: Despite the increasing amount of research on Web-based mental health interventions with proven efficacy, high attrition rates decrease their effectiveness. Continued process evaluations should be performed to maximize the target population's engagement. Google Analytics has been used to evaluate various health-related Web-based programs and may also be useful for Web-based mental health programs. OBJECTIVE: The objective of our study was to evaluate WalkAlong.ca, a youth-oriented mental health web-portal, using Google Analytics to inform the improvement strategy for the platform and to demonstrate the use of Google Analytics as a tool for process evaluation of Web-based mental health interventions. METHODS: Google Analytics was used to monitor user activity during WalkAlong's first year of operation (Nov 13, 2013-Nov 13, 2014). Selected Google Analytic variables were overall website engagement including pages visited per session, utilization rate of specific features, and user access mode and location. RESULTS: The results included data from 3076 users viewing 29,299 pages. Users spent less average time on Mindsteps (0 minute 35 seconds) and self-exercises (1 minute 08 seconds), which are important self-help tools, compared with that on the Screener tool (3 minutes 4 seconds). Of all visitors, 82.3% (4378/5318) were desktop users, followed by 12.7 % (677/5318) mobile phone and 5.0% (263/5318) tablet users. Both direct traffic (access via URL) and referrals by email had more than 7 pages viewed per session and longer than average time of 6 minutes per session. The majority of users (67%) accessed the platform from Canada. CONCLUSIONS: Engagement and feature utilization rates are higher among people who receive personal invitations to visit the site. Low utilization rates with specific features offer a starting place for further exploration of users in order to identify the root cause. The data provided by Google Analytics, although informative, can be supplemented by other evaluation methods (ie, qualitative methods) in order to better determine the modifications required to improve user engagement. Google Analytics can play a vital role in highlighting the preferences of those using Web-based mental health tools.

Centre for Applied Research in Mental Health and Addictions Faculty of Health Sciences Simon Fraser University Burnaby BC Canada

Centre for Health Evaluation and Outcome Sciences St Paul's Hospital Vancouver BC Canada

Department of Pharmacology Faculty of Medicine Masaryk University Brno Czech Republic

Department of Psychiatry University of British Columbia Vancouver BC Canada

School of Population and Public Health University of British Columbia Vancouver BC Canada

Zobrazit více v PubMed

Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012 Dec;12(6):745–764. doi: 10.1586/erp.12.67. PubMed DOI

Griffiths KM, Christensen H. Review of randomised controlled trials of Internet interventions for mental disorders and related conditions. Clin Psychol. 2006 Mar;10(1):16–29. doi: 10.1080/13284200500378696. DOI

Rice SM, Goodall J, Hetrick SE, Parker AG, Gilbertson T, Amminger GP, Davey CG, McGorry PD, Gleeson J, Alvarez-Jimenez M. Online and social networking interventions for the treatment of depression in young people: a systematic review. J Med Internet Res. 2014 Sep 16;16(9):e206. doi: 10.2196/jmir.3304. v16i9e206 PubMed DOI PMC

Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. 2014 Feb;13(1):4–11. doi: 10.1002/wps.20083. doi: 10.1002/wps.20083. PubMed DOI PMC

Eysenbach G. The law of attrition. J Med Internet Res. 2005 Mar 31;7(1):e11. doi: 10.2196/jmir.7.1.e11. v7e11 PubMed DOI PMC

Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res. 2009 Apr 24;11(2):e13. doi: 10.2196/jmir.1194. v11i2e13 PubMed DOI PMC

Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012 Jun;32(4):329–342. doi: 10.1016/j.cpr.2012.02.004.S0272-7358(12)00027-X PubMed DOI

Calear AL, Christensen H, Mackinnon A, Griffiths KM. Adherence to the MoodGYM program: outcomes and predictors for an adolescent school-based population. J Affect Disord. 2013 May;147(1-3):338–344. doi: 10.1016/j.jad.2012.11.036.S0165-0327(12)00803-8 PubMed DOI

Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, Hohagen F, Hautzinger M, Klein JP. The association between adherence and outcome in an Internet intervention for depression. J Affect Disord. 2018 Dec 15;229:443–449. doi: 10.1016/j.jad.2017.12.028.S0165-0327(17)32480-1 PubMed DOI

Bruwer B, Sorsdahl K, Harrison J, Stein DJ, Williams D, Seedat S. Barriers to mental health care and predictors of treatment dropout in the South African Stress and Health Study. Psychiatr Serv. 2011 Jul;62(7):774–781. doi: 10.1176/ps.62.7.pss6207_0774. 62/7/774 PubMed DOI PMC

Byrne L, Schoeppe S, Bradshaw J. Recovery without autonomy: Progress forward or more of the same for mental health service users? Int J Ment Health Nurs. 2018 Feb 15; doi: 10.1111/inm.12446. PubMed DOI

Davis MJ, Addis ME. Predictors of attrition from behavioral medicine treatments. Ann Behav Med. 1999 Dec;21(4):339–349. doi: 10.1007/BF02895967. PubMed DOI

Pampallona S, Bollini P, Tibaldi G, Kupelnick B, Munizza C. Patient adherence in the treatment of depression. Br J Psychiatry. 2002 Feb;180:104–109. PubMed

O'Brien HL, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci Technol. 2008 Apr;59(6):938–955. doi: 10.1002/asi.20801. DOI

National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Centers for Disease Control and Prevention. [2018-02-26]. Types of evaluation https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf .

Bhattacharya A, Hopkins S, Sallis A, Budd EL, Ashiru-Oredope D. A process evaluation of the UK-wide Antibiotic Guardian campaign: developing engagement on antimicrobial resistance. J Public Health (Oxf) 2017 Jun 01;39(2):e40–e47. doi: 10.1093/pubmed/fdw059.fdw059 PubMed DOI

Steckler A, Linnan L, editors. Process evaluation for public health interventions and research. San Francisco, CA: Jossey-Bass; 2004. Dec 01,

Volker D, Zijlstra-Vlasveld MC, Brouwers EPM, van DFCM. Process Evaluation of a Blended Web-Based Intervention on Return to Work for Sick-Listed Employees with Common Mental Health Problems in the Occupational Health Setting. J Occup Rehabil. 2017 Jun;27(2):186–194. doi: 10.1007/s10926-016-9643-4. 10.1007/s10926-016-9643-4 PubMed DOI PMC

Crutzen R, Roosjen JL, Poelman J. Using Google Analytics as a process evaluation method for Internet-delivered interventions: an example on sexual health. Health Promot Int. 2013 Mar;28(1):36–42. doi: 10.1093/heapro/das008. das008 PubMed DOI

Clark D, Nicholas D, Jamali HR. Evaluating information seeking and use in the changing virtual world: the emerging role of Google Analytics. Learn Pub. 2014 Jul 01;27(3):185–194. doi: 10.1087/20140304. DOI

Kirk M, Morgan R, Tonkin E, McDonald K, Skirton H. An objective approach to evaluating an internet-delivered genetics education resource developed for nurses: using Google Analytics™ to monitor global visitor engagement. J Res Nurs. 2012 Oct 29;17(6):557–579. doi: 10.1177/1744987112458669. DOI

McCloskey EV, Johansson H, Harvey NC, Compston J, Kanis JA. Access to fracture risk assessment by FRAX and linked National Osteoporosis Guideline Group (NOGG) guidance in the UK-an analysis of anonymous website activity. Osteoporos Int. 2017 Dec;28(1):71–76. doi: 10.1007/s00198-016-3696-2.10.1007/s00198-016-3696-2 PubMed DOI

Danaher BG, Boles SM, Akers L, Gordon JS, Severson HH. Defining participant exposure measures in Web-based health behavior change programs. J Med Internet Res. 2006;8(3):e15. doi: 10.2196/jmir.8.3.e15. v8i3e15 PubMed DOI PMC

Makkar SR, Howe M, Williamson A, Gilham F. Impact of tailored blogs and content on usage of Web CIPHER - an online platform to help policymakers better engage with evidence from research. Health Res Policy Syst. 2016 Dec 01;14(1):85. doi: 10.1186/s12961-016-0157-5. 10.1186/s12961-016-0157-5 PubMed DOI PMC

Vona P, Wilmoth P, Jaycox LH, McMillen JS, Kataoka SH, Wong M, DeRosier ME, Langley AK, Kaufman J, Tang L, Stein BD. A web-based platform to support an evidence-based mental health intervention: lessons from the CBITS web site. Psychiatr Serv. 2014 Nov 01;65(11):1381–1384. doi: 10.1176/appi.ps.201300512. 1897771 PubMed DOI PMC

Thabrew H, Stasiak K, Merry S. Protocol for Co-Design, Development, and Open Trial of a Prototype Game-Based eHealth Intervention to Treat Anxiety in Young People With Long-Term Physical Conditions. JMIR Res Protoc. 2017 Sep 22;6(9):e171. doi: 10.2196/resprot.7250. v6i9e171 PubMed DOI PMC

Lindgaard G, Fernandes G, Dudek C, Brown J. Attention web designers: You have 50 milliseconds to make a good first impression! Behav Inform Technol. 2006 Mar;25(2):115–126. doi: 10.1080/01449290500330448. DOI

Rainie L, Smith A. Pew Research Center. Washington, D.C: 2013. Oct 18, [2017-07-28]. 35 % of those ages 16 older own tablet computers http://www.pewinternet.org/files/old-media//Files/Reports/2013/PIP_Tablets%20and%20e-readers%20update_101813.pdf .

Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Prof Psychol:Res Pract. 2011;42(6):505–512. doi: 10.1037/a0024485. DOI

Crutzen R, de Nooijer J, Brouwer W, Oenema A, Brug J, de Vries N. Effectiveness of online word of mouth on exposure to an Internet-delivered intervention. Psychol Health. 2009 Jul 01;24(6):651–661. doi: 10.1080/08870440802521094.905685535 PubMed DOI

Kaushik A. Web analytics: an hour a day. Indianapolis, IN: Wiley Publishing, Inc; 2007.

O data, where art thou? Revolutionizing data sharing to advance our sustainability goals through smart chemical innovation