A Mixed Methods Evaluation of a Pilot Resilience Training Course on Stress Management

Shawn T. Mason, PhD

Behavioral Science & Advanced Analytics, Johnson & Johnson Health and Wellbeing Solutions, One Johnson & Johnson Plaza, New Brunswick, NJ 08933, USA; E-mail: smason5@its.jnj.com

INTRODUCTION

The impact of stress has critical implications for both individuals and organizations.^1,2 Stress for individual health has been associated with heart disease, diabetes, decreased immune functioning, as well as mental health disorders such as depression and anxiety.³ At the organizational level, stress has been associated with increased rates of medical errors and employee burnout, retention and turnover.^4,5 The most recent American Psychological Association (APA) survey on stress found that although overall stress was stable from 2016 to 2017, individuals were more likely to report experiencing the effect of stress.⁶ In 2015, the survey found that 24% of adults reported extreme levels of stress compared to 18% in 2014. In this same survey, 34% of the adult report that their stress increased over the past year, while only 16% report decreased stress in the past year.⁷ These statistics suggest that stress is a critical challenge.

Although multiple approaches have been developed to intervene on stress, various levels of impact have been observed.^8,9,10 Individuals typically start by learning to identify types of stressors and associated cues and triggers which helps improve awareness and may help to plan for and or prepare for expected situations and future stress.¹¹ Mental exercises such as meditation and mindfulness can also be beneficial. Magnetic Resonance Imaging (MRI) scans show that after an eight-week course of mindfulness practice, the brain’s “fight or flight” center appears to shrink.¹² Other benefits have included reduced symptoms of anxiety and depression,¹² improvements of emotional regulation,¹² learning, and memory¹³ and decrease the amygdala response.¹² A 2012 study by Keller et al found that individuals who experience high-levels of stress and perceive that stress affects their health are at greater risk for poor health and mortality outcomes.¹⁴ Cognitive framing is thus an important technique used to change how individuals understand and experience stress. “Eustress”,¹⁵ for example, is used to describe and emphasize the positive benefits of stress. Overall, an abundance of research has been dedicated to understanding the effects of minimization or removal of stress on health and behavior. However, the relationships between sustained stress, the redefinition of stress, and strategic recovery within workplace settings are still not fully understood. A 2011 study used the randomized controlled trial to assess the effect of a Stress Management and Resiliency Training (SMART) program on 25 women diagnosed with breast cancer. The study found out the brief training enhanced Resilience and Quality of Life (QoL) and decreased stress and anxiety at 12-weeks among treatment arm but not the control arm. This type of research led to the current study to evaluate a resilience training program on community participants.¹⁶

The objective of the study was to determine the impact of a resilience to stress approach, where resilience is the ability to  reframe or to bounce back from stressful experiences. The Johnson & Johnson Human Performance Institute (HPI) Resilience training program takes this approach through recalibrating mindset and changing resilience supportive behavior. Short-term follow-up was employed to determine initial viability of the intervention. The quantitative component sought to understand changes in specific behaviors and in measures of participants’ health and wellbeing, while the qualitative component sought to understand how participants may have recalibrated their mindsets to change their behaviors.

METHODS

Participants

Research participants were 30 community members located in Central Florida, USA. Individuals registered as part of the Lake Nona Life Project Community health study¹⁷ were recruited via email. This study was reviewed and approved by the Advarra (formerly Chesapeake) Internal Review Board (IRB). Each participant provided written consent. Inclusion criteria included: 1) Adults (18 years or older), 2) speak, read, and understand English fluently, 3) willing to complete baseline assessments at least 10-days prior to the training course, 4) willing to attend a full day training program at the Lake Nona Institute during the designated training date, 5) willing to provide email and phone number as a contact method, 6) able to comprehend and follow the requirements of the study, 7) have a valid e-mail address. 8) able to provide informed consent (IC), 9) willing and able to comply with all study procedures for the duration of the study. There were no limitations or exclusion criteria beyond inclusion criteria. Participant means age was 52.8-years (SD=9.0). Demographic characteristics are displayed in Table 1.

Design

This study employed a single arm, interventional, mixed methods (qualitative & quantitative), pre-post design. A control group was not included in order to balance the program’s stage of development and scope of the project. Instead, qualitative interviews were intended to help explain or refute potential quantitative outcomes. Therefore, no extraneous variables were measured via survey besides demographic characteristics and outcome measures. Participants completed baseline surveys between 21 to 3-days  prior to the intervention and again 30-days afterward. Telephone qualitative semi-structured interviews were conducted on a rolling basis between 3-68-days post-intervention.

Intervention

The resilience course is a 1-day training program that employs the use of adult learning principles and experiential learning design to create an immersive face-to-face experience for participants. The rationale for creating a 1-day intervention is consistent with broader efforts to support brief behavioral interventions (e.g., minimize participant burden, reduce attrition, and potentially enhance engagement). Further, there is a widespread need for programs that have not only evidence for effectiveness, but also can be systematically delivered on a large scale to populations suffering from stress and burnout. General employee and health provider populations, in particular, have received much attention in this regard. This study examines the initial level of evidence for such a program. The primary objectives of the course are to:

1. Understand the definition of resilience and its     relationship to performance and wellbeing.
2. Recognize the difference between different types of stress.
3. Learn how to strategically recover to manage stress and     improve resilience.

The course is led by a certified facilitator where participants are guided through the course content through a variety of individual and group activities, self-reflection, introspection, and concept presentations. Included with the course is a resilience assessment that participants use as a tool to help them understand their current level of resilience, oscillation patterns of high and low stress, resilience behaviors, and mental framework for understanding stress called stress mindset. This enables participants to identify their areas of strength and opportunity. By recognizing the gaps between current resilience and desired resilience levels, and connecting with a sense of purpose or meaningful direction in life, participants could develop a more comprehensive action plan. This action plan is designed by each participant and defines specific action steps (rituals), accountability, and setback management tactics that they use to better manage stress and improve resilience.

Lastly, participants received access to the HPI mobile app as a digital sustainability tool to help them track their progress in their 90-day journey. The app is designed to provide timely tips and strategies to help manage stress by using recovery exercise strategically, mindfulness, and promoting a deeper connection to having a meaningful sense of purpose and direction in life.

Materials and Measures

Survey measures included the Perceived Stress Scale (PSS),¹⁸ public health surveillance wellbeing (PHS-WB),¹⁹ Rand Medical Outcome Survey (MOS) SF-36 (SF-36),²⁰ health and productivity questionnaire (HPQ),²¹ Work Productivity and Activity Impairment (WPAI),²² and Brief Cope Scale (BCS).²³ PSS included 5 items, where a total score ranged 0-40 was calculated and three stratifications were made: low stress (score 0-13), moderate stress (score 14-26), high stress (27-40). PHS-WB included 10 items and a total score ranged 0-40 was calculated as the higher score indicates better wellbeing. Items from 5 domains of SF-36 were included in the survey: Role limitations due to physical health, energy, emotional wellbeing, social functioning, and general health. Domain scores ranged 0-100 were calculated as the higher score indicates better health/wellbeing. BCS includes 28 items, and the domain scores ranged 2-8 were calculated for 14 domains: self distraction, active coping, denial, substance use, use of emotional support, use of instrumental support, behavioral disengagement, venting, positive reframing, planning, humor, acceptance, religion, and self blame. To measure job performance, 3 items from HPQ were used to calculate absolute presenteeism and relative presenteeism, with scores, ranged 0-100, and 0-10 respectively and a higher score indicating better performance. One modified item from WPAI was used to measure presenteeism due to low energy: “During the past seven days, how much did your low (or less than optimal) energy levels affect your productivity while you were working?” (0=Low energy had no effect on my work; 10=Low energy completely prevented me from working). For qualitative, a telephone-based 30-minute semi-structured interview was used to obtain data, and analysis methods were used based on grounded theory.²⁴ A grounded theory seeks to explain the main concern of participants and how that concern is resolved or processed: the focus is on patterns of behavior.

Analysis

Descriptive data were computed on frequencies and percentages of each level of the demographic variable and PSS stratification. Means and  Standard Deviations (SDs) were computed for continuous outcome variables at baseline and follow-up. For statistical analysis, Generalized Estimating Equations (GEE) were used to test if the probability of falling low, moderate or high category are different at baseline vs., follow-up on PSS stratification since this is a categorical variable. Paired sample tests were used for other study outcomes since they are all continuous variables, to test if the mean change from baseline to follow-up are the difference from zero. Ninety five percent of confidence intervals on the change were also computed. Analysis was conducted using SPSS V24 (Chicago, IL, USA). A p-value less than 00.05 was considered statistically significant since the study sample size is small (n=30).

For the qualitative analysis, after open coding using the constant comparison method,²⁴ a review of the initial patterns emerging from the first 12 interviews was conducted. A recent study of a course using the same educational principles had emerged both the main concern and a key concept (‘core category’). During the review, we sought and established ‘emergent fit’ with these concepts.²⁵

Once the core category is identified, future data collected is specific and intended to saturate the core category and related categories. The act of collecting specific data is ‘theoretical sampling’, whilst the act of analyzing specific data is ‘selective coding’.²⁶ The interview guide was therefore refined to facilitate theoretical sampling and the remaining ¹⁸ participants interviewed and the data analyzed. Throughout analysis, memos are written about concepts and the relationships between them. Table 2 shows the development of the concepts theoretical and personal frameworks for action emerging from initial ideas about course design, designed opportunity for change and personal opportunity for change. The ideas captured in memos were separated, compared and sorted to reveal the overall shape of the theory (Figure 1: Theory of Personal reform: Living the perspective).

Figure 1. Shape of Theory

The main concern of the participants in this study is to take the opportunity of the course to better their lives, specifically to improve their resilience, or at least to address the consequences of stress in their lives. The theory explains the degree to which the theoretical framework for action embedded in the course design becomes incorporated into a person’s mindset and the impact of the application of that integration, such as it is.

RESULTS

Survey Data

All 30 participants completed both the baseline and follow-up surveys. Please see scores in Table 3. PSS stratification was changed significantly (p<0.001). Most participants moved from moderate or high stress at baseline to low or moderate stress at 1-month follow-up. PSS total score did not decrease significantly. The PHS-WB total score and physical health subscale improved significantly (p<0.05). For BCS, “Self-distraction” (p=0.022), “Planning” (p=0.001) and “Humor” (p=0.035) subscale changed significantly. Four out of 5 subscales in SF-36 were significantly improved as well. Participants reported less role limitations due to physical health, more energy, better emotional and social functioning at 1-month after the training (p<0.01). Participants also reported better general health score, but not significantly. Finally, there was no significant change on the three presenteeism scores from HPQ and WPAI.

Qualitative Data

Participants experience the course as a process of guided introspection. Success is the degree to which participants were able to develop their resilience perspective using a recalibration process of reframing, solutioning and sensitizing, and the extent to which they are able to apply this perspective to process stress events (living the perspective). Success is contingent upon taking the opportunity of the course and introspecting to recalibrate perspectives and co-varies with an individual’s capacity to introspect (Figure 1). Key behaviors in the continual processing of future stress events are: (i) sensitizing; recognizing that stress is being experienced, its source and selecting an appropriate method to process the stress and (ii) integrating; taking the time to introspect to identify the causes of stress and to practice methods of reducing stress.

Three types of participants emerged from the analysis. ‘Seekers’ have an established resilience perspective, seek to work at the frontiers of resilience and seek an edge to their knowledge. They are beyond the course. ‘Seed planters’ remain stuck in a status quo either because they have made silent decisions to prioritize a different perspective or because a different perspective dominates introspection and compromises recalibration. For this group, the seeds of ideas planted during the course may grow into new perspectives. ‘Reformers’ introspect, recalibrate their mindsets to some degree and apply their revised perspectives to better process their stress by sensitizing and integrating.

DISCUSSION

The purpose of this pilot study was to test the impact of a novel, brief intervention designed to enhance individual level resilience to stress. The novelties of the intervention include the 1-day group format and the combination of techniques employed. Quantitative results show improvements across various measures of stress and functioning which were observed up to a 30-day period following the course. They included stress perception, wellbeing and quality of life across multiple domains, and enhancement in some specific coping mechanism uses. Qualitative results helped to explain the phenomena participants experienced from a process perspective. The model derived from interviews suggests that participants needed to have an adequate level of openness to the intervention (e.g., taking the opportunity) and the capacity to introspect about their lives and life direction. Recalibration of their mindset and beliefs about stress, as well as their capacity to reform were pre-cursors to integrating the course material into their lives.

Among the principles taught in the course, there are some notables. First, oscillation, the principle used to organize life activities as they relate to stress and integrates this concept of eustress. By not avoiding or removing stress, the focus can be placed on how a sequence of stress paired with strategic recovery, or “oscillation”, can be beneficial to performance and overall wellbeing.²⁷ Oscillation, also conceptualized as the strategic recovery has shown positive outcomes of recovery in various shapes or durations. For example, Bergoulgnan et al found evidence that “microbursts of activity during the day improve energy level, mood, and fatigue level while maintaining usual levels of cognitive function.” The microbursts of physical activity were conceptualized to be strategic recovery breaks throughout a sedentary workday condition.²⁸

Secondly is the purpose in life, another factor that has been shown to positively impact recovery and performance. A study led by Schaefer et al²⁹ found that “purpose in life predicts both health and longevity suggesting that the ability to find meaning from life’s experiences, especially when confronting life’s challenges, may be a mechanism underlying resilience.” Having a purpose in life may motivate reframing stressful situations to deal with them more productively, thereby facilitating recovery from stress and trauma. In turn, enhanced ability to recover from negative events may allow a person to achieve or maintain a feeling of greater purpose in life over time.³⁰ Other studies have indicated that a greater sense of meaning and purpose in life have been shown to positively impact people’s emotional recovery from negative situations, sensitivity to pain, and ability to heal from illness more effectively and quickly.³¹

In regard to stress mindset, research has also shown different stress mindsets impact health perceptions, Here, individuals who embraced a “stress-is-enhancing” mindset reported having better health than those who endorse a “stress-is-debilitating” mindset: specifically, respondents reported fewer symptoms of depression and anxiety while also reporting higher levels of energy. Additionally, both workplace performance and overall satisfaction with life were positively correlated with a “stress-is-enhancing” mindset.³²

Results from this study suggest consistency with findings in the research on separate intervention topics, and further suggest the integration of these topics in a brief, group format may provide benefit to individuals experiencing stress in life in the short-term. Primary limitations of this study are characteristics of typical pilot research in that no control group was used and the follow-up was a brief 30-days. In addition, the stress-mindset measure was a planned screener and primary outcome for the study. However, due to technical errors this survey could not be included. Last, the generalization of the program effectiveness needs to be cautious given the representativeness of the study sample. Future research on this intervention will include a control condition, have a longer follow, and may be applied in specific settings where stress is prevalent, such as health care delivery.

ACKNOWLEDGMENT

The authors would like to thank the staff at the Lake Nona Institute for organizing and facilitating the project: Kathy Brown, Kaley Miller, and Gloria Caulfield. Significant other contributions were appreciated from Luis Arroyo and Anna Begelfer in study planning and execution.

CONFLICTS OF INTEREST

Grounded Solutions Ltd. received fees for research consultancy from Johnson & Johnson Health and Wellbeing Solutions. Remaining authors were employees of Johnson & Johnson during the time of this study.

1. Schneiderman N, Ironson G, Siegel SD. Stress and  ealth: Psychological, behavioral, and biological determinants. Annual Review of Clinical Psychology. 2005; 1: 607-628. doi: 10.1146/annurev.clinpsy.1.102803.144141

2. Workplace Stress. The American Institute of Stress Website: https://www.stress.org/workplace-stress/. Accessed July 19, 2018.

3. Five Things You Should Know About Stress. National Institute of Mental Health. Website: https://www.nimh.nih.gov/health/publications/stress/index.shtml. Accessed July 19, 2018.

4. Waterman AD, Garbutt J, Hazel E, et al. The emotional impact of medical errors on practicing physicians in the United States and Canada. Jt Comm J Qual Patient Saf. 2007; 33(8): 1-10. doi: 10.1016/S1553-7250(07)33050-X

5. Mosadeghrad AM. Occupational stress and turnover intention: Implications for nursing management. Int J Health Policy Manag. 2013; 1(2): 169-176. doi: 10.15171%2Fijhpm.2013.30

6. Stress in America: The State of our Nation. Stress in America^TM Survey. American Psychological Association. 2017.

7. Stress in America: The Impact of Discrimination. American Psychological Association. 2016.

8. Lamontagne AD, Keegel T, Louie AM, Ostry A, Landsbergis PA. A systematic review of the job-stress intervention evaluation literature, 1990-2005. Int J Occup Environ Health. 2007; 13(3): 268-80. doi: 10.1179/oeh.2007.13.3.268

9. Goyal M, Singh S, Sibinga EM, et al. Meditation programs for psychological stress and wellbeing: A systematic review and meta-analysis. JAMA Intern Med. 2014; 174(3): 357-368. doi: 10.1001/jamainternmed.2013.13018

10. Fjorback LO, Arendt M, Ørnbøl E, Fink P, Walach H. Mindfulness-based stress reduction and mindfulness-based cognitive therapy: A systematic review of randomized controlled trials. Acta Psychiatr Scand. 2011; 124(2): 102-119. doi: 10.1111/j.1600-0447.2011.01704.x

11. Elder JP, Ayala GX, Harris S. Theories and intervention approaches to health-behavior change in primary care. Am J Prev Med. 1999; 17(4): 275-284. doi: 10.1016/s0749-3797(99)00094-x

12. Holzel BK, Carmody J, Vangel M, et al. Mindfulness practice leads to increases in regional brain gray matter density. Psychiatry Res. 2011; 191(1): 36-43. doi: 10.1016%2Fj.pscychresns.2010.08.006

13. Mrazek MD, Franklin MS, Phillips DT, Baird B, Schooler JW. Mindfulness training improves working memory capacity and GRE performance while reducing mind wandering. Psychol Sci. 2013; 24(5): 776-781. doi: 10.1177/0956797612459659

14. Keller A, Litzelman K, Wisk LE, et al. Does the perception that stress affects health matter? The association with health and mortality. Health Psychol. 2012; 31(5): 677-684. doi: 10.1037/a0026743

15. Kupriyanov R, Zhdanov R. The eustress concept: problems and outlooks. World Journal of Medical Sciences. 2014; 11(2): 179-185. doi: 10.5829/idosi.wjms.2014.11.2.8433

16. Loprinzi C, Prasad K, Schroeder D, Sood A. Stress management and resilience training (SMART) program to decrease stress and enhance resilience among breast cancer survivors: A pilot randomized clinical trial. Clinical Breast Cancer. 2011; 11(6): 364-368. doi: 10.1016/j.clbc.2011.06.008

17. Lake Nona Life Project Community Health Study. Website: http://www.liveworkparticipate.com/. Accessed July 19, 2018.

18. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983; 24(4): 385-396. doi: 10.2307/2136404

19. Bann CM, Kobau R, Lewis MA, Zack MM, Thompson WW. Development and psychometric evaluation of the public health surveillance wellbeing scale. Qual Life Res. 2012; 21(6): 1031-1043. doi: 10.1007/s11136-011-0002-9

20. Ware Jr JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Med Care.1992; 473-483.

21. Kessler RC, Barber C, Beck AL, et al. The world health organization health and work performance questionnaire (HPQ). J Occup Environ Med. 2003; 45(2): 156-174. doi: 10.1097/01.jom.0000052967.43131.51

22. Reilly MC, Zbrozek AS, Dukes EM. The validity and reproducibility of a Work Productivity and Activity Impairment instrument. Pharmacoeconomics. 1993; 4(5): 353-365. doi: 10.2165/00019053-199304050-00006

23. Carver CS. You want to measure coping but your protocol’s too long: Consider the brief COPE. Int J Behav Med. 1997; 4(1): 92-100. doi: 10.1207/s15327558ijbm0401_6

24. Glaser B. Theoretical Sensitivity. Mill Valley, CA, USA: Sociology Press. 1978.

25. Glaser B. Doing Grounded Theory: Issues and Discussions. Mill Valley, CA, USA: Sociology Press. 1998.

26. Glaser B, Strauss A. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, USA: Aldine Publishing. 1967. doi: 10.4324/9780203793206

27. Loehr J. Toughness Training for Life. New York, USA: Plume by Penguin Group.1993.

28. Bergouignan A, Legget KT, De Jong N, et al. Effect of frequent interruptions of prolonged sitting on self-perceived levels of energy, mood, food cravings and cognitive function. Int J Behav Nutr Phys Act. 2016; 13(1): 113. doi: 10.1186/s12966-016-0437-z

29. Schaefer SM, Morozink Boylan J, Van Reekum CM, et al. Purpose in life predicts better emotional recovery from negative stimuli. PLoS One. 2013; 8(11): e80329. doi: 10.1371/journal.pone.0080329. eCollection 2013

30. Kim ES, Strecher VJ, Ryff CD. Purpose in life and use of preventive health care services. Proc Natl Acad Sci U S A. 2014; 111(46): 16331-16336. doi: 10.1073/pnas.1414826111

31. Hill PL, Turiano NA. Purpose in life as a predictor of mortality across adulthood. Psychol Sci. 2014; 25(7): 1482-1486. doi: 10.1177/0956797614531799

32. Crum A, Salovey P, Achor S. Rethinking stress: The role of mindsets in determining the stress response. J Pers Soc Psychol. 2013; 104(4): 716-733. doi: 10.1037/a0031201