|
| |
 |
|
| ARTICLE |
|
| Year : 2021 | Volume
: 19
| Issue : 1 | Page : 26-36 |
|
A System Innovation Perspective on the Potential for Scaling Up New Psychological Interventions for Refugees
Aniek Woodward1, Marjolein A Dieleman2, Egbert Sondorp3, Bayard Roberts4, Daniela C Fuhr4, Peter Ventevogel5, Marit Sijbrandij6, Jacqueline E.W Broerse2, On behalf of the STRENGTHS consortium7
1 KIT Royal Tropical Institute, KIT Health, Amsterdam; Athena Institute, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, The Netherlands
2 Athena Institute, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, The Netherlands
3 KIT Royal Tropical Institute, KIT Health, Amsterdam, The Netherlands
4 London School of Hygiene and Tropical Medicine, Faculty of Public Health and Policy, Department of Health Services Research and Policy, London, UK
5 Public Health Section, Division of Resilience and Solutions, United Nations High Commissioner for Refugees, Geneva, Switzerland
6 Department of Clinical, Neuro and Developmental Psychology, World Health Organization Collaborating Center for Research and Dissemination of Psychological Interventions, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, The Netherlands
| Date of Submission | 29-Sep-2020 |
| Date of Decision | 20-Dec-2020 |
| Date of Acceptance | 16-Feb-2021 |
| Date of Web Publication | 31-Mar-2021 |
Correspondence Address:
Aniek Woodward
KIT Royal Tropical Institute, KIT Health, Mauritskade 64, 1092 AD, Amsterdam
The Netherlands
 Source of Support: None, Conflict of Interest: None
DOI: 10.4103/INTV.INTV_29_20
In recent years, a range of brief protocolised psychological interventions like Problem Management Plus have been developed. Such “scalable psychological interventions” are meant to be delivered by nonspecialists which can greatly increase access to psychological therapies for people affected by adversity, including forced displacement. However, embedding new interventions into mainstream practices is challenging. Novel interventions can remain in the research phase for a long time or stop altogether, which minimises their intended impact and reach. In this conceptual paper we propose a “system innovation perspective” on scaling up new psychological interventions for refugees and argue that existing mental health systems often need to change to integrate new interventions in a sustainable way. We present a conceptual framework, which includes ideas on cycles of deepening (learning by doing), broadening (repeating and linking), and scaling up (embedding) and the multilevel and constellation perspective. This framework has been operationalised in our scalability research as part of the STRENGTHS study in which we increase our understanding of the opportunities for scaling up four new psychological interventions in eight countries hosting Syrian refugees, including in Europe (Germany, Netherlands, Sweden and Switzerland) and the Middle East (Egypt, Jordan, Turkey and Lebanon).
Key implications for practice
- Researchers and practitioners involved in implementing and evaluating scalable psychological interventions such as Problem Management Plus should consider the inclusion of scalability assessments to increase understanding about the potential for integrating such innovations into mainstream services.
- A system innovation perspective views scaling up as the integration of an innovation into mainstream practices and suggests that mental health systems commonly need to change in order to effectively adopt new interventions, allowing them to reach their desired impact at scale and in a sustainable way.
- An improved understanding of the scalability of novel psychological interventions, including the potential (systemic) barriers and facilitators for scaling up, will provide essential knowledge for those involved in decision-making, implementation and evaluation of the further scale up of such interventions.
Keywords: mental health, Problem Management Plus (PM+), psychological interventions, refugees, scalability, system innovation perspective, theory
How to cite this article:
Woodward A, Dieleman MA, Sondorp E, Roberts B, Fuhr DC, Ventevogel P, Sijbrandij M, Broerse JE, On behalf of the STRENGTHS consortium. A System Innovation Perspective on the Potential for Scaling Up New Psychological Interventions for Refugees. Intervention 2021;19:26-36 |
How to cite this URL:
Woodward A, Dieleman MA, Sondorp E, Roberts B, Fuhr DC, Ventevogel P, Sijbrandij M, Broerse JE, On behalf of the STRENGTHS consortium. A System Innovation Perspective on the Potential for Scaling Up New Psychological Interventions for Refugees. Intervention [serial online] 2021 [cited 2023 Jun 7];19:26-36. Available from: http://www.interventionjournal.org//text.asp?2021/19/1/26/312716 |
| Introduction | |  |
Limited financial resources and scarcity of qualified mental health professionals remain obstacles to meeting the growing global mental health burden. In order to increase coverage of mental health services, a range of scalable psychological interventions have been developed for use in communities affected by adversity (e.g. poverty, violence, humanitarian emergency, conflict and post-conflict; World Health Organization [WHO], 2019). Such communities are commonly at high risk of psychological suffering and struggle to access mental health and psychosocial support (MHPSS) services (WHO, 2019). These new psychological interventions are potentially highly scalable adaptations of existing evidence-based psychological treatments (WHO, 2010). They are adapted so that they can be delivered in a shorter timeframe through guided self-help or with the support of trained and supervised nonspecialists (Murray et al., 2019; WHO, 2019). This shifting of tasks from mental health professionals to nonspecialists and beneficiaries aims to increase access to psychological therapies by requiring less resources and specialist skills than traditional one-to-one therapy provided by a mental health professional (Bennett-Levy et al., 2010; Murray et al., 2019; WHO, 2019).
Problem Management Plus
Problem Management Plus (PM+) is an example of a scalable psychological intervention based on task-shifting. Initial results on the feasibility and effectiveness of scalable psychological interventions are promising. For example, definitive trials with individual forms of PM+ (one-to-one support) showed significant reductions in psychological distress within PM+ participants compared to those who received enhanced usual care after 3 months follow-up in Kenya and Pakistan (Bryant et al., 2017; Rahman et al., 2016). A group version of PM+ showed feasibility and acceptability in Nepal (Rahman et al., 2019) and significant reductions in anxiety and depressive symptoms at 3 months in Pakistan (Rahman et al., 2019). More research is being done in other contexts and for other populations, including for Syrian refugees in various high- and middle-income countries as part of the Syrian REfuGees MeNTal HealTH Care Systems (STRENGTHS) study.
STRENGTHS Study
The war in Syria has led to over 6 million refugees in the Middle East and Europe. The psychological wellbeing of Syrian refugees has been impacted by experiencing extreme stressors before, during and after their flight from Syria and facing obstacles for accessing MHPSS services in host countries (Hassan et al., 2015; Hendrickx et al., 2019; Kiselev et al., 2020; Quosh et al., 2013). Besides the effectiveness and cost-effectiveness of PM+, the STRENGTHS study also examines the scalability of PM+ and related interventions amongst Syrian refugees in some of Syria’s neighbouring countries and European countries (Sijbrandij et al., 2017).
STRENGTHS will run from 2017–2022 and is a consortium of universities and nongovernmental organisations. It evaluates four novel psychological interventions in eight different countries: PM+ individual in The Netherlands (de Graaff et al., 2020a, b) and Switzerland (Kiselev et al., 2020), PM+ group in Jordan (Akhtar et al., 2020) and Turkey (Fuhr et al., 2019, 2020b; Uygun et al., 2020), Early Adolescent Skills for Emotions (EASE) in Lebanon (Brown et al., 2019) and Step-by-Step (SbS) in Germany, Sweden and Egypt (Burchert et al., 2019). [Table 1] provides further details on these interventions.
Challenges of Scaling Up
The uptake of novel psychological interventions into mainstream practices, however, is challenging (Ibrahim et al., 2020; Murray et al., 2014). Previous studies on implementation and scaling up processes of health interventions (Bulthuis et al., 2020; Esponda et al., 2019; Milat et al., 2015; Wakida et al., 2018), including those that involve task-shifting and community-based approaches (Javadi et al., 2017; Kok et al., 2015; Palas et al., 2013; Patel et al., 2011; Scott et al., 2018; Thornicroft et al., 2010) have run into substantial barriers. Developing and fine-tuning the innovation and a mismatch between the innovation and the adopting organisation can delay diffusion of the new intervention (Rotmans & Loorbach, 2010). There may be resistance to change and various institutional barriers may need to be lifted (Broerse & Bunders, 2010; van der Ham et al., 2013). As a result, novelties can remain in the research phase for a prolonged period or stop altogether. These scaling up challenges may explain the “painfully slow” translation of knowledge about effective ways to prevent and treat mental health problems into routine practices and visible reductions in the global mental health burden (Patel et al., 2018). How can we prevent PM+ and similar novel interventions from falling into this implementation gap?
Scalability Evaluation
We believe a starting point is assessing the scalability of new interventions. A recent review highlighted the need for improved incorporation of scalability considerations alongside effectiveness trials (Zamboni et al., 2019). Assessing scalability can increase understanding of the “suitability” or “potential” for scaling up an evidence-based intervention (Milat et al., 2013, 2020; WHO & ExpandNet, 2011) and is perceived as an important part of the scaling up process (Kohl & Cooley, 2003; WHO & ExpandNet, 2010). Scaling up is the process of institutionalising a new intervention into existing systems (Chibanda, 2018; Eaton et al., 2018; Simmons & Shiffman, 2007; Ventevogel et al., 2011; Yamey, 2011), which is required for making its expansion sustainable (Simmons & Shiffman, 2007).
The scaling up of novel psychological interventions for Syrian refugees involves some level of integration into existing humanitarian and governmental MHPSS systems in STRENGTHS host countries. With the Syrian civil conflict ongoing since 2011, we can assume Syrian refugees will remain in their host countries for many more years, or even permanently. This means the MHPSS systems of host countries need to be able to cater for a large number of Syrian refugees for a prolonged period of time. Conflict-affected populations have a relatively high prevalence (22.1%) of common mental health disorders (depression, anxiety and posttraumatic stress disorder; Charlson et al., 2019). Refugees also experience barriers to accessing mental health services in host countries, such as: language, sociocultural, financial, lack of awareness, help-seeking behaviours, stigma and a mismatch between the local health system and perceived needs of refugees (Hendrickx et al., 2019; Kiselev et al., 2020; Satinsky et al., 2019). As a result, refugees commonly have high unmet MHPSS needs.
Novel interventions like PM+ that are provided in the native language of Syrians and through lay providers from Syria may be an acceptable and accessible option in addition to existing MHPSS services. Whether it is feasible to integrate a novel intervention in different settings in such a way that it improves access to MHPSS for Syrian refugees in the long run, however, is an important question which will be addressed in our research on the scalability of new psychological interventions like PM+. We believe a radical perspective on scaling up is required to more fully understand its challenges and the possible ways to overcome them.
In this conceptual paper we start by making a case for adopting a system innovation perspective on scaling up. This is followed by an explanation of our conceptual framework, including definitions of key concepts. Finally, we describe how we have operationalised and are applying our conceptual framework in ongoing scalability research as part of STRENGTHS.
| The Case for a System Innovation Perspective | | |
A growing number of frameworks are available on the scaling up and scalability of innovative evidence-based interventions (Milat et al., 2015, 2020; Tabak et al., 2012; Zamboni et al., 2019). There are many commonalities between frameworks, with most having similar theoretical origins in diffusion of innovation theory, knowledge transfer theory, organisational theory and political science theory (Tabak et al., 2012; Zamboni et al., 2019). Previous frameworks have recommended enhancing compatibility of new interventions with prospective adopting delivery systems. Innovative interventions, however, are designed to solve problems in society and existing systems; thus their integration into mainstream services will by definition require some level of change in the old ways of thinking (culture), organising (structure) and doing (practices). For example, new interventions like PM+ that involve task-shifting will require system change from specialised to community-based providers, which would be in line with transformational shifts happening at the global level (Patel et al., 2018). And online interventions like SbS will involve a shift from face-to-face to online MHPSS, which builds upon the increasing use of digital technologies globally. In other words, as these innovations instigate system change they are likely to run into “systemic” barriers. For this reason we use the system innovation perspective as our overarching framework for examining scalability. This perspective recognises that commonly existing systems need changing to scale up an innovation in such a way that it achieves the positive long-term impact for which it was initially developed (Broerse & Grin, 2017, p. 286).
Theory of Change (ToC) maps, logic models and process evaluations frameworks can improve understanding of the causal assumptions underpinning a complex intervention or project and how its inputs may lead to certain long-term outcomes (De Silva et al., 2014; Moore et al., 2015). These rather linear models, however, can oversimplify the interactions between a novel health intervention and its context of implementation (i.e. existing systems). The system innovation perspective, on the other hand, embraces the complexity and uncertainty of this interaction. As is common in systems thinking, it perceives systems as open, complex, dynamic, interlinked, resistant to change, history and context dependent (Loorbach, 2010). While ToC maps and process evaluations can be used as part of research on scaling up potential (e.g. for developing a vision for system innovation), they should, in our view, not be the dominant lens for understanding the scaling up process. This is in line with previous calls on the need for complexity and systems thinking when studying the scaling up of health innovations (Atun, 2012; Greenhalgh & Popoutsi, 2018; Paina & Peters, 2012), including task-shifting for mental health (Javadi et al., 2017).
System innovation research is diverse and has been applied in different disciplines and sectors (Loorbach et al., 2017), including health systems (Broerse & Bunders, 2010; Broerse & Grin, 2017). While this research perspective has become increasingly global (Loorbach et al., 2017), to the best of our knowledge, it has not yet been used with regards to scaling up of MHPSS for refugees. Adopting a system innovation lens may offer new and useful insights about the scalability of new interventions like PM+, including the possible barriers to scaling up and ways to address them, and how to implement new interventions in such a way that they strengthen and transform local mental health systems in the long-term.
| A System Innovation Perspective on Scaling Up | | |
Scaling Up and Scalability
Based on the system innovation literature, we perceive scaling up as the process of embedding an innovation into existing delivery systems (Rotmans & Loorbach, 2010). We conceive scalability as the potential for such an integration. We define innovation according to Roger’s diffusion of innovation theory as: “an idea, practice or object that is perceived to be new by an individual or unit of adoption” (Rogers, 2003, p. 12). Systems are a “nested phenomenon”, meaning various subsystems could be identified within a system (van Raak, 2010, p. 57). For example, a mental health system may exist of a health and social system, and the health system can again comprise private, public and nongovernmental subsystems and so on.
In the system innovation literature, various mechanisms are distinguished on how to move an experiment from the “protected” environment of the research level to the “real world” system level. Based on case studies from the healthcare sector, the steering mechanisms “deepening, broadening and scaling up” were proposed (van den Bosch, 2010; Van den Bosch & Rotmans, 2008). Our conceptual framework ([Figure 1]) combines the multilevel perspective (Geels, 2002), the constellation perspective (de Haan, 2010; van Raak, 2010) and the mechanisms broadening–deepening–scaling up (Van den Bosch & Rotmans, 2008). | Figure 1 System Innovation Perspective on Scaling Up New Psychological Interventions. Note. It combines the multilevel perspective (Geels, 2002), the constellation perspective (de Haan, 2010; van Raak, 2010) and mechanisms of broadening–deepening–scaling up (van den Bosch & Rotmans, 2008).
Click here to view |
Multilevel and Constellation Perspective
The multilevel perspective helps to understand the complex and dynamic relationships between the niche (experimental setting), regime (mainstream practices, culture and structure) and landscape (external context) level. Related to the multilevel perspective is the constellation perspective. The constellation perspective more elaborately describes the regime and relabels it the dominant constellation. In this article we use the latter term because this is more “neutral”, particularly in the context of conflict-affected populations (i.e. “regime” has been used to describe authoritarian governments). The constellation concept elaborates on “structure, culture and practices” and highlights the role of “actors” within complex societal systems like health care (Loorbach et al., 2017).
The landscape level encompasses the broader societal trends and contexts of social change, such as demographics and cultural changes or other developments like “economic growth, wars, emigration, broad political coalitions” (Geels, 2002, p. 1260). Landscape changes are usually slow and may put pressure on the system (Geels, 2002). This external context is believed to be beyond the influence of system actors, at least in the short run (Rotmans & Loorbach, 2010, p. 24; van den Bosch & Rotmans, 2008). For this reason, previous research on system innovation in mental health care recommended to focus on the strategies for scaling up niche level experiments, whilst connecting these to broader landscape trends (van der Ham et al., 2013).
The constellation level involves the dominant set of structure, culture and practices of the existing social system (these set of elements are further explained below; de Haan, 2010). These elements “both define and fulfil a function in a larger social system in a specific way” (van Raak, 2010, p. 52). Complex systems, like the health system, could be perceived as having various subsystems (constellations); “each of which is concerned with a specific aspect of the health system’s overall functioning” (van Raak & de Haan, 2017, p. 48). The mental health system could be seen as a constellation of the health system; with a focus on promoting and improving the emotional, psychological and social wellbeing of populations:- Culture is described as the “set of values, perceptions and interpretive frames − relating to or relevant for the system − that are shared by most of the involved actors” (van Raak, 2010, p. 55). It involves the “ways of thinking, mental models and perceptions” (van der Ham et al., 2013, p. 127) or “how it is perceived” (de Haan, 2010, p. 41).
- Structure is defined as the “physical, economic, legal, financial, organisational and power structures that facilitate and/or constrain the behaviour of involved actors” (van Raak, 2010, p. 55). In other words, it refers to “how it works” (de Haan, 2010, p. 41) or “how people organise the things they do, either physically, institutionally or financially” (van der Ham et al., 2013, p. 127).
- Practice involves “actual actions (operations) undertaken within the constellations, which are relevant for the functioning of the constellation” (van Raak, 2010, p. 54). In short it is “what people actually do” (van der Ham et al., 2013, p. 127). Practice has also been described as the “result of the behaviour of actors” (van Raak, 2010, p. 54). Interactions between people and behaviour of actors will be numerous; analysis therefore focuses on practices that are “typical for the subsystem” (van Raak, 2010, p. 54), such as description of the care pathway (i.e. from patient formulation of complaint to release and final check-up; van Raak, 2010).
- Actors are defined as “individuals or organised groups that act as a unity” and are seen as related but not part of the system (van Raak, 2010, p. 55).
Structure and culture can be seen as structuring elements. They are different in nature but have the same role in the system. Both are “shaped by the practices in which actors engage” (the agency of actors) and may “limit what actors can or want to do in these practices” (van Raak, 2010, p. 53). The latter gives constellations stability − constellations are reproduced by actors − and thus lead to system resilience.
At the niche level, actors experiment with innovations in a protected space (Geels, 2002; Schot, 1998). Innovative experiments are generally “sheltered from mainstream competition” and may function as “proto-markets” for the development of market experiments, and eventually system shifts (Schot & Geels, 2008, p. 539). Experimental settings are important locations for learning processes and for building the social networks to support innovations (Geels, 2002). Niches emerge because actors feel that the current constellation is not able to satisfactorily solve a problem or realise an opportunity that would fulfil the function of the system in a better way. For example, the current mental health system is not able to guarantee refugees access to care and thereby cannot fulfil its purpose of delivering quality healthcare to those in need. Hence, some actors feel the need to try out new practices, and these may require new cultures and structures.
System innovation theory suggests these three concepts (niche, constellation and landscape) are related through a nested hierarchy, with the dominant constellations embedded within landscapes and niches within the dominant constellations. Within the multilevel perspective there is no linear causality, meaning there is no simple cause and effect relationship for radical system change (Geels, 2002). A transition (i.e. change in the dominant culture, structure and practices) usually stretches over one or two generations (Rotmans et al., 2001) and can be influenced at three levels: top-down by larger sociopolitical trends, self-change within the system and bottom-up through the scaling up of innovative experimental solutions (van Raak, 2010). Alignment of trajectories within and between the levels will produce transitions, according to the multilevel perspective (Rotmans & Loorbach, 2010). Likewise “windows of opportunity” for transition can be created through tensions, which are understood as powerful changes in trajectories, or shifts in the landscape that put pressure on the constellation (Geels, 2002; Rotmans & Loorbach, 2010). This is when innovations can break out of the niche level and, over time, instigate change at the level of the dominant constellation and eventually landscape level.
Mechanisms: Deepening, Broadening, Scaling Up
However, the stability of dominant constellations prevents successful integration of innovations that entail a practice, culture and structure that is considerably different from mainstream practices, culture and structure. To facilitate this integration process van den Bosch & Rotmans (2008, p. 42) have developed three types of mechanisms:- deepening involves learning processes which take place in a relatively protected space at local level;
- broadening entails linking and repeating experiments in different contexts and
- scaling up is the process in which innovative experiments become mainstream (Johansen & van den Bosch, 2017; van der Ham et al., 2013).
Broadening and scaling up are similar to “horizontal” (replication) and “vertical” scaling (institutionalisation) from the WHO/ExpandNet framework on scaling up health interventions (Simmons et al., 2007). Cycles of deepening, broadening and scaling up could be perceived as contributing to the integration of innovative experiments into mainstream services (van den Bosch & Rotmans, 2008, p. 42).
Scaling up has been identified as the most challenging of the three mechanisms (i.e. deepening, broadening and scaling up). Identifying the (systemic) barriers and facilitators is considered important for scaling up experiments in the system innovation literature (Johansen & van den Bosch, 2017; van den Bosch & Rotmans, 2008) and mental health literature alike (Eaton et al., 2011; Murray et al., 2014; WHO, 2008). Factors influencing scaling up may occur at landscape, constellation and niche level. Systemic barriers and facilitators are factors rooted in the dominant culture, structure and practice. Political will is an example of a systemic influencing factor for scale up, which may act as a barrier (if powerful actors lack political will) or a facilitator (if important actors have sufficient political will) depending on the circumstance.
| Next Steps: Application of Our Conceptual Framework | | |
The conceptual framework presented in this article is guiding our ongoing scalability research as part of the STRENGTHS study. Guiding research questions are: “How can new psychological interventions for refugees be integrated into existing MHPSS systems in such a way that they increase access to MHPSS for refugees in the long-term?” and “What are the potential barriers and facilitators (at multilevel and culture–structure–practice) of such an integration?” Answers to these questions will help us to provide recommendations on the next steps with regards to scaling up the brief psychological interventions being evaluated in each study site.
We have operationalised the above conceptual framework in the STRENGTHS study into three phases: (1) visioning − developing desired impact pathways, (2) systems analysis and (3) identification of (systemic) barriers and facilitators for scaling up. Each phase is described in more detail below.
Phase 1: Visioning − Developing Desired Impact Pathways
To develop desired impact pathways we organised ToC workshops in three STRENGTHS’ countries (Turkey, Netherlands and Lebanon; Fuhr et al., 2020a,b). Key local actors were invited to participate in 1-day ToC workshops on scaling up. During these workshops ToC maps were developed, which outline the causal pathways through which PM+ and related interventions are expected to achieve their impact within the constraints of the systems. The cross-country ToC map outlined two interdependent causal pathways (policy and financing; health services) for scaling up new psychological interventions like PM+, including various intermediate and long-term outcomes as well as perceived interventions and assumptions for achieving these outcomes and its eventual impact (Fuhr et al., 2020a).
Phase 2: Systems Analysis
To gain insight into the current health system in the participating countries we conducted a desk study. The desk study consists of ongoing rapid appraisals of the mental health systems in the eight project countries of STRENGTHS. Its methodology is explained in detail elsewhere (Fuhr et al., 2020c). In brief, the rapid appraisals involve desk-based reviews of available data and published literature on mental health system inputs (leadership, financing, facilities and services, medicines, health workforce and information) and process outcomes (care pathway, access and coverage, quality and safety), which are complemented by existing qualitative data collected by STRENGTHS partners as part of their formative work to support the cultural and local adaptation of the new interventions. These rapid appraisals aim to improve understanding of the current health systems in STRENGTHS countries and its responsiveness to the mental health needs of Syrian refugees. Preliminary results indicate that Syrian refugees experience various barriers in accessing MHPSS services in existing host systems, especially acceptability challenges (e.g. stigma, culture, language). Besides these appraisals, we conducted a systematic review on scaling up MHPSS interventions for refugees and other populations affected by humanitarian crises (Troup et al., 2021). In this review, we found limited evidence on the scaling up of MHPSS interventions for populations affected by crisis. Existing scaling up efforts focused on expansion rather than integration of new interventions and reported many barriers to scaling up, particularly with regards to health sector capacity (Troup et al., 2021).
Phase 3: Identification of (Systemic) Barriers and Facilitators
Experienced and anticipated (systemic) barriers and facilitators are identified through the desk study (see above) as well as interviews with relevant stakeholders. In-depth primary qualitative data are being collected through semi-structured interviews in the STRENGTHS study countries with purposively selected key informants (researchers, implementers, policy makers), MHPSS providers (including PM+ supervisors and helpers), Syrian refugees receiving the new interventions and if feasible their family members. In these interviews we explore perceptions of local and national actors on the possible (systemic) barriers and facilitators for integrating new psychological interventions in existing systems, including the roles of various organisations and individuals in the integration process. [Table 2] gives an overview of some of the topics that are being explored in semi-structured interviews, including how they link to the conceptual framework. | Table 2 Topics Being Explored as Part of Semi-Structured Interviews Using PM+ as Example
Click here to view |
Topic guides slightly vary depending on the type of psychological intervention being tested (PM+ individual, PM+ group, EASE and SbS; [Table 1]), country and background of interviewee and are, where possible, integrated into the process evaluations of the clinical trials to avoid duplication and increase efficiency. Semi-structured interviews will be analysed thematically in two stages. The first stage will be an inductive analysis, coding the data based upon emerging themes. The second stage will be to deductively organise the identified data themes in relation to the key elements of the conceptual framework. Results from these semi-structured interviews will be triangulated with findings from the desk research and ToC workshops.
Within STRENGTHS there are various teams with specific tasks. Several teams consist of clinical trial and intervention specialists, which are shaping the new practices and doing the day-to-day management (including troubleshooting) of the clinical trials. Other teams involve system innovation specialists who are monitoring the structure and culture implications of the new practice and its relation with the dominant constellation, as well as identifying issues related to broadening and scaling up and developing strategies to overcome such issues. As there is variation between the four psychological interventions being tested as part of STRENGTHS ([Table 1]) and there are differences between the eight study sites (including high- and middle-income countries in Europe and the Middle East) we anticipate the scaling up potential to vary across intervention type and site. This variation may aid in our reflection on the notion of scalability in the context of novel psychological interventions for refugees at the end of our scalability research.
Potential limitations
Potential limitations of applying the framework are mainly challenges of a conceptual analytic nature due to the complexity of the subject of study: systems. Firstly, challenges may be faced in demarcating the system when conducting a system analysis. Systems are made up of subsystems and interact with other systems. It is therefore complicated to set boundaries and to determine which actors and which interactions and interdependencies are part of the system.
Another analytic complexity is in characterising interventions as systemic innovations. Systemic innovations comprise a mix of coherent, innovative and interrelated actions that, together, may move society to a tipping point, beyond which a more desirable constellation is realised, one that is better able to fulfil the purpose of the system. In this sense, systemic innovations can be thought of as “game changers” − or as triggers of radical changes within the dominant culture, structure and practice of the system (Avelino et al., 2019). Many interventions tend to be merely system optimisers, unable to transform culture, structure and practice significantly. It is complicated to select the appropriate mix of coherent, innovative and interrelated actions. This requires reflexive learning and action research cycles, analysing systems dynamics and adapting actions accordingly. However, such an approach does not fit easily with the more structured experimental settings preferred in health sciences to attain disciplinary scientific standards (such as randomised controlled trials and quasi-experiments).
Furthermore, in the relatively short timeframe of most projects (4 to 5 years), it is difficult to measure systemic impact. Systemic change is a long-term process typically spanning at least one but usually more decades. It is quite complex to formulate impact indicators that prove you are heading towards system transformation. The often-used validated questionnaires are insufficient. It requires a co-creation process to develop a tailor-made monitoring and evaluation framework.
| Conclusion | | |
Establishing evidence on the effectiveness of new psychological interventions will not automatically lead to their uptake into mainstream practices. Scaling up can be challenging and slow, particularly due to systemic barriers. System innovation ideas on cycles of deepening (learning by doing), broadening (repeating and linking), and scaling up (embedding) and the multilevel and constellation perspective offers us an explanation on how new interventions like PM+ can potentially break into and improve access to MHPSS systems for larger groups of refugees. This may implicate that “old” ways of organising, thinking and doing need to be exchanged by “new” ones. Attention for refugee mental health has the potential to lead to structural changes in mental health care delivery if, from the onset, implementation of novel interventions is geared towards integration in existing systems. System innovation may not just be beneficial to refugees but also to their host communities. An improved understanding about the scalability (i.e. potential for scaling up) of novel psychological interventions will provide essential knowledge for the next steps in scaling up such interventions. We encourage other MHPSS researchers and practitioners to contribute to this effort of generating knowledge about the scalability of novel psychological interventions and the usefulness of theoretical approaches like the system innovation perspective.
Financial support and sponsorship
This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme Societal Challenges under grant agreement No 733337. The content of this article reflects only the authors’ views and the European Community is not liable for any use that may be made of the information contained therein.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Akhtar A., Giardinelli L., Bawaneh A., Awwad M., Naser H., Whitney C., Jordans M. D., Sijbrandij M., Bryant R.A., on behalf of the STRENGTHS consortium. (2020). Group problem management plus (gPM+) in the treatment of common mental disorders in Syrian refugees in a Jordanian camp: Study protocol for a randomized controlled trial. BMC Public Health, 20, 390. https://doi.org/10.1186/s12889-020- 08463-5  |
| 2. | |
| 3. | Avelino F., Wittmayer J. M., Pel B., Weaver P., Dumitru A., Haxeltine A., Kemp R., Jørgenson M. S., Bauler T., Ruijsink S., O’Riordon T. (2019). Transformative social innovation and (dis)empowerment. Technological Forecasting and Social Change, 145, 195-206. https://doi.org/10.1016/j.techfore.2017.05.002 |
| 4. | Bennett-Levy J., Richards D., Farrand P. (2010). Low intensity CBT interventions: A revolution in mental health care. In Bennett-Levy J., Richards D., Farrand P., Christensen H., Griffiths K., Kavanagh D., Klein B., Lau M., Proudfoot J., Ritterband L., White J., Williams C. (Eds.), Oxford guide to low intensity CBT interventions. Oxford University Press. |
| 5. | Broerse J. E. W., Bunders J. F. G. (2010). Transitions in health systems: Dealing with persistent problems. VU University Press. |
| 6. | Broerse J. E. W., Grin J. (2017). The future of health systems: Beyond the persistence of contemporary challenges. In Broerse J.E.W., Grin J. (Eds.), Toward sustainable transitions in healthcare systems. Routledge. |
| 7. | Brown F. L., Steen F., Taha K., Aoun M., Bryant R. A., Jordans M. J. D., Malik A., van Ommeren M., Abualhaija A., Said Agel I., Ghatesheh M., Habashneh R., Sijbrandij M., El Chammay R., Watts S., Akthar A., on behalf of the STRENGTHS consortium. (2019). Early Adolescent Skills for Emotions (EASE) intervention for the treatment of psychological distress in adolescents: Study protocol for randomised controlled trials in Lebanon and Jordan. Trials, 20, 545. https://doi.org/10.1186/s13063-019-3654-3 |
| 8. | Bryant R. A., Schafer A., Dawson K. S., Anjuri D., Mulili C., Ndogoni L., Koyiet P., Sijbrandij M., Ulate J., Harper Shehadeh M., Hadzi-Pavlovic D., van Ommeren M. (2017). Effectiveness of a brief behavioural intervention on psychological distress among women with a history of gender-based violence in urban Kenya: A randomised clinical trial. PLoS Medicine, 14(8), e1002371. https://doi.org/10.1371/journal.pmed.1002371 |
| 9. | Bulthuis S., Kok M., Raven J., Dieleman M. (2020). Factors influencing the scale-up of public health interventions in low- and middle-income countries: A qualitative systematic literature review. Health Policy and Planning, 35 (2), 219-234. https://doi.org/10.1093/heapol/czz140 |
| 10. | Burchert S., Alkneme M. S., Bird M., Carswell K., Cuijpers P., Hansen P., Heim E., Harper Shehadeh M., Sijbrandij M., Van’t Hof E., Knaevelsrud C. (2019). User-centered app adaptation of a low-intensity e-mental health intervention for Syrian refugees. Frontiers in Psychiatry, 9, 663. https://doi.org/10.3389/fpsyt.2018.00663 |
| 11. | Carswell K., Harper-Shehadeh M., Watts S., van’t Hof E., Abi Ramia J., Heim E., Wenger A., van Ommeren M. (2018). Step-by-Step: A new WHO digital mental health intervention for depression. mHealth, 4, 34. https://doi.org/10.21037/mhealth.2018.08.01 |
| 12. | Charlson F., van Ommeren M., Flaxman A., Comett J., Whiteford H., Saxena S. (2019). New WHO prevalence estimates of mental disorders in conflict settings: A systematic review and meta-analysis. The Lancet, 394(10194), 240-248. |
| 13. | Chibanda D. (2018). Programmes that bring mental health services to primary care populations in the international setting. International Review of Psychiatry, 30(6), 170-181. |
| 14. | Dawson K. S., Bryant R. A., Harper M., Kuowei Tay A., Rahman A., Schafer A., van Ommeren M. (2015). Problem Management Plus (PM+): A WHO transdiagnostic psychological intervention for common mental health problems. World Psychiatry, 14(3), 354-357. https://doi.org/10.1002/wps.20255 |
| 15. | Dawson K., Watts S., Carswell K., Shehadeh M., Jordans M., Bryant R., Miller K. E., Malik A., Brown F. L., Servili C., van Ommeren M. (2019). Improving access to evidence-based interventions for young adolescents: Early Adolescent Skills for Emotions (EASE). World Psychiatry, 18(1), 105-107. |
| 16. | de Graaff A. M., Cuijpers P., Acarturk C., Bryant R., Burchert S., Fuhr D. C., Huizink A. C., de Jong J., Kieft B., Knaevelsrud C., McDaid D., Morina M., Park A., Uppendahl J., Ventevogel P., Whitney C., Wiedemann N., Woodward A., Sijbrandij M. (2020a). Effectiveness of a peer-refugee delivered psychological intervention to reduce psychological distress among adult Syrian refugees in the Netherlands: Study protocol. European Journal of Psychotraumatology, 11(1), 1694347. https://doi.org/10.1080/20008198.2019.1694347 |
| 17. | de Graaff A. M., Cuijpers P., McDaid D., Park A., Woodward A., Bryant R. A., Fuhr D. C., Kieft B., Minkenberg E., Sijbrandij M. (2020b). Peer-provided Problem Management Plus (PM+) for adult Syrian refugees: A pilot randomised controlled trial on effectiveness and cost-effectiveness. Epidemiology and Psychiatric Sciences, 29, e162. https://doi.org/10.1017/S2045796020000724 |
| 18. | de Haan J. (2010). Towards transition theory. [Doctoral dissertation, Erasmus University]. Rotterdam. |
| 19. | De Silva M., Breuer E., Lee L., Asher L., Chowdhary N., Lund C., Patel V. (2014). Theory of change: A theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials, 15, 267. |
| 20. | Eaton J., McCay L., Semrau M., Chatterjee S., Baingana F., Araya R., Ntulo C., Thornicroft G., Saxena S. (2011). Scale up of services for mental health in low-income and middle-income countries. The Lancet, 378 (9802), 1592-1603. https://doi.org/10.1016/S0140-6736(11)60891-X |
| 21. | Eaton J., Gureje O., De Silva M., Sheikh T. L., Ekpe E. E., Abdulaziz M., Muhammad A., Akande Y., Onukogu U., Onyuku T., Abdulmalik J., Fadahunsi W., Nwefoh E., Cohen A. (2018). A structured approach to integrating mental health services into primary care: Development of the Mental Health Scale Up Nigeria intervention (mhSUN). International Journal of Mental Health Systems, 12, 11. https://doi.org/10.1186/s13033-018- 0188-0 |
| 22. | Esponda G. M., Hartman S., Qureshi O., Sadler E., Cohen A., Kakuma R. (2019). Barriers and facilitators of mental health programmes in primary care in low-income and middle-income countries. Lancet Psychiatry. https://doi.org/10.1016/S2215-0366(19) 30125-7 |
| 23. | Fuhr D. C., Acarturk C., McGrath M., Ilkkursun Z., Sondorp E., Sijbrandij M., Ventevogel P., Cuijpers P., McKee M., Roberts B. (2019). Treatment gap and mental health service use among Syrian refugees in Sultanbeyli, Istanbul: A cross-sectional survey. Epidemiology and Psychiatric Sciences, 29, e70. https://doi.org/10.1017/S2045796019000660 |
| 24. | Fuhr D. C., Acarturk C., Sijbrandij M., Brown F. L., Jordans M., Woodward A., McGrath M., Sondorp E., Ventevogel P., Ikkursun Z., El Chammay R., Cuijpers P., Roberts B. (2020a). Planning the scale up of brief psychological interventions using theory of change. BMC Health Services Research, 20, 801. https://doi.org/10.1186/s12913-020-05677-6 |
| 25. | Fuhr D. C., Acarturk C., Uygun E., McGrath M., Ikkursun Z., Kaykha S., Sondorp E., Sijbrandij M., Ventevogel P., Cuijpers P., Roberts B., on behalf of the STRENGTHS consortium. (2020b). Pathways towards scaling up Problem Management Plus in Turkey: A theory of change workshop. Conflict and Health, 14, 22. https://doi.org/10.1186/s13031-020-00278-w |
| 26. | Fuhr D. C., Roberts B., Woodward A., Sondorp E., Sijbrandij M., de Graaff A. M., Balabanova D., on behalf of the STRENGTHS consortium. (2020c). Health systems responsiveness to the mental health needs of forcibly displaced persons. In Bozorgmehr K., Roberts B., Razum O., Biddle L. (Eds.), Health policy and systems responses to forced migration. Springer International Publishing. |
| 27. | Geels F. W. (2002). Technological transitions as evolutionary reconfiguration processes: A multi-level perspective and a case-study. Research Policy, 31(8/9), 1257-1274. |
| 28. | |
| 29. | Hassan G., Kirmayer L. J., Mekki-Berrada A., Quosh C., El Chammay R., Deville-Stoetzel J. B., Youssef A., Jefee-Bahloul H., Barkeel-Oteo A., Coutts A., Song S., Ventevogel P. (2015). Culture, context and the mental health and psychosocial wellbeing of Syrians: A review for mental health and psychosocial support staff working with Syrians affected by armed conflict. |
| 30. | Hendrickx M., Woodward A., Fuhr D. C., Sondorp E., Roberts B. (2019). The burden of mental disorders and access to mental health and psychosocial support services in Syria and among Syrian refugees in neighboring countries: A systematic review. Journal of Public Health, 1-12. https://doi.org/10.1093/pubmed/fdz097 |
| 31. | Ibrahim N., Thompson D., Nixdorf R., Kalha J., Mpango R., Moran G., Mueller-Stierlin A., Ryan G., Mahlke C., Shamba D., Puschner B., Repper J., Slade M. (2020). A systematic review of influences on implementation of peer support work for adults with mental health problems. Social Psychiatry and Psychiatric Epidemiology, 55. https://doi.org/10.1007/s00127-019- 01739-1 |
| 32. | Javadi D., Feldhaus I., Mancuso A., Ghaffar A. (2017). Applying systems thinking to task shifting for mental health using lay providers: A review of the evidence. Global Mental Health (Cambridge, England), 4, e14. https://doi.org/10.1017/gmh.2017.15 |
| 33. | Johansen F., van den Bosch S. (2017). The scaling-up of Neighbourhood Care: From experiment towards a transformative movement in healthcare. Futures, 89, 60-73. |
| 34. | Kiselev N., Pfaltz M., Haas F., Schick M., Kappen M., Sijbrandij M., De Graaff A. M., Bird M., Hansen P., Ventevogel P., Fuhr D. C., Schnyder U., Morina N. (2020). Structural and socio-cultural barriers to accessing mental healthcare among Syrian refugees and asylum seekers in Switzerland. European Journal of Psychotraumatology, 11 (1), 1717825. https://doi.org/10.1080/20008198.2020.1717825 |
| 35. | Kohl R., Cooley L. (2003). Scaling up: A conceptual and operational framework. Management Systems International. |
| 36. | Kok M., Dieleman M., Taegtmeyer M., Broerse J., Kane S., Ormel H., Tijm M., de Koning K. (2015). Which intervention design factors influence performance of community health workers in low- and middle-income countries? A systematic review. Health Policy and Planning, 30, 1207-1227. https://doi.org/10.1093/heapol/czu126 |
| 37. | Loorbach D. (2010). Transition management for sustainable development: A prescriptive, complexity-based governance framework. Governance, 23(1), 161–183. |
| 38. | Loorbach D., Frantzeskaki N., Avelino F. (2017). Sustainability transitions research: Transforming science and practice for societal change. Annual Review of Environment and Resources, 42, 566-626. |
| 39. | Milat A., King L., Bauman A., Redman S. (2013). The concept of scalability: Increasing the scale and potential adoption of health promotion interventions into policy and practice. Health Promotion International, 28(3), 285-298. |
| 40. | |
| 41. | Milat A., Lee K., Conte K., Grunseit A., Wolfenden L., van Nassau F., Orr N., Sreeram P., Bauman A. (2020). Intervention scalability assessment tool: A decision support tool for health policy makers and implementers. Health Research Policy and Systems, 18, 1-17. https://doi.org/10.1186/s12961-019- 0494-2 |
| 42. | Moore G. F., Audrey S., Barker M., Bond L., Bonell C., Hardeman W., Moore L., O’Cathain A., Tinati T., Wight D., Baird J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ, 350, h1258. https://doi.org/10.1136/bmj.h1258 |
| 43. | Murray L., Haroz E., Pullmann M., Dorsey S., Kane J., Augustinavicius J., Lee C., Bolton P. (2019). Under the hood: Lay counsellor element use in a modular multi-problem transdiagnostic intervention in lower resource countries. Cognitive Behavioural Therapy, 12. https://doi.org/doi:10.1017/S1754470(18000144) |
| 44. | Murray L., Tol W., Jordans M., Zangana G. S., Amin A. M., Bolton P., Bass J., Bonilla-Escobar F. J., Thornicroft G. (2014). Dissemination and implementation of evidence based, mental health interventions in post conflict, low resource settings. Intervention, 12(suppl 1), 94-112. https://doi.org/10.1097/WTF 0000000000000070 |
| 45. | Paina L., Peters D. H. (2012). Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy and Planning, 27, 365-373. https://doi.org/10.1093/heapol/czr054 |
| 46. | Palas S., Minhas D., Pérez-Escamilla R., Taylor L., Curry L., Bradley E. (2013). Community health workers in low- and middle-income countries: What do we know about scaling up and sustainability? American Journal of Public Health, 103(7), e74-e82. |
| 47. | Patel V., Chowdhary N., Rahman A., Verdeli H. (2011). Improving access to psychological treatments: Lessons from developing countries. Behaviour Research and Therapy 49, 523-528. |
| 48. | Patel V., Saxena S., Lund C., Thornicroft G., Baingana F., Bolton P., Chisholm D., Collins P.Y., Cooper J.L., Eaton J., Herrman H. (2018). The Lancet Commission on global mental health and sustainable development. Lancet, 392 (10157), 1553-1598. http://dx.doi.org/10.1016/ S0140-6736(18)31612-X |
| 49. | |
| 50. | Rahman A., Hamdani S. U., Awan N. R., Bryant R. A., Dawson K. S., Khan M. F., van Ommeren M. (2016). Effect of a multicomponent behavioral intervention in adults impaired by psychological distress in a conflict-affected area of Pakistan: A randomized clinical trial. JAMA, 316 (24), 2609-2617. https://doi.org/10.1001/jama.2016.17165 |
| 51. | Rahman A., Khan M., Hamdani S., Chiumento A., Akhtar P., Nazir H., Nisar A., Masood A., Ud Din I., Khan N., Bryant R. A., Dawson K. S., Sijbrandij M., Wang D., van Ommeren M. (2019). Effectiveness of a brief group psychological intervention for women in a post-conflict setting in Pakistan: A single-blind, cluster, randomised controlled trial. Lancet, 393(10182), 1733-1744. https://doi.org/10.1016/S0140-6736(18)32343-2 |
| 52. | Rogers E. (2003). Diffusion of innovations (5th ed.). Free Press. |
| 53. | Rotmans J., Loorbach D. (2010). Towards a better understanding of transitions and their governance: A systematic and reflexive approach. In Grin J., Rotmans J., Schot J. (Eds.), Transitions to sustainable development: New directions in the study of long-term transformative change. Routledge. |
| 54. | |
| 55. | Satinsky E., Fuhr D. C., Woodward A., Sondorp E., Roberts B. (2019). Mental health care utilisation and access among refugees and asylum seekers in Europe: A systematic review. Health Policy. https://doi.org/10.1016/j.healthpol.2019.02.007 |
| 56. | Schot J. (1998). The usefulness of evolutionary models for explaining innovation. The case of the Netherlands in the nineteenth century. History and Technology, an International Journal, 14 (3), 173-200. https://doi.org/10.1080/07341519808581928 |
| 57. | Schot J., Geels F. W. (2008). Strategic niche management and sustainable innovation journeys: Theory, findings, research agenda, and policy. Technology Analysis & Strategic Management, 20(5), 537-554. |
| 58. | Scott K., Beckham S. W., Gross M., Pariyo G., Rao K. D., Cometto G., Perry H. B. (2018). What do we know about community-based health worker programs? A systematic review of existing reviews on community health workers. Human Resources for Health, 16, 39. https://doi.org/10.1186/s12960-018-0304-x |
| 59. | Sijbrandij M., Acarturk C., Bird M., Bryant R. A., Burchert S., Carswell K., de Jong J., Dinesen C., Dawson K., El Chammay R., van Ittersum L., Jordans M., Knaevelsrud C., McDaid D., Miller K., Morina N., Park A., Roberts B., van Son Y., …Cuijpers P. (2017). Strengthening mental health care systems for Syrian refugees in Europe and the Middle East: Integrating scalable psychological interventions in eight countries. European Journal of Psychotraumatology, 8(Suppl 2). https://doi.org/10.1080/20008198.2017.1388102 |
| 60. | Simmons R., Shiffman J. (2007). Scaling up health service innovations: A framework for action. In Scaling up health service delivery: from pilot innovations to policies and programmes 1-30. World Health Organisation. |
| 61. | Simmons R., Fajans P., Ghiron L. (2007). Scaling up health service delivery: From pilot innovations to policies and programmes. World Health Organization. |
| 62. | Tabak R. G., Khoong E. C., Chambers D. A., Brownson R. C. (2012). Bridging research and practice: Models for dissemination and implementation research. American Journal of Preventative Medicine, 43 (3), 337-350. https://doi.org/10.1016/j.amepre.2012.05.024 |
| 63. | Thornicroft G., Alem A., Dos Santos R. A., Barley E., Drake R., Gregorio G., Hanlon C., Ito H., Latimer E., Law A., Mari J., McGeorge P., Padmavati R., Razzouk D., Semrau M., Setoya Y., Thara R., Wondimagegn D. (2010). WPA guidance on steps, obstacles and mistakes to avoid in the implementation of community mental health care. World Psychiatry, 9(2), 67-77. |
| 64. | Troup J., Fuhr D., Woodward A., Sondorp E., Roberts B. (2021). Barriers and facilitators for scaling up mental health and psychosocial support interventions in low- and middle-income countries for populations affected by humanitarian crises: A systematic review of the literature. International Journal of Mental Health Systems, 15, 5. https://doi.org/10.1186/s13033-020- 00431-1 |
| 65. | Uygun E., Ilkkursun Z., Sijbrandij M., Aker A. T., Bryant R., Cuijpers P., Fuhr D. C., de Graaff A. M., de Jong J., McDaid D., Morina N., Park A. L., Roberts B., Ventevogel P., Yurtbakan T., Acarturk C., STRENGHTS consortium. (2020). Protocol for a randomized controlled trial: peer-to-peer Group Problem Management Plus (PM+) for adult Syrian refugees in Turkey. Trials, 21(1), 283. https://doi.org/10.1186/s13063-020-4166-x |
| 66. | van den Bosch S. (2010). Transition experiments. Exploring societal changes towards sustainability. [Doctoral dissertation, Erasmus University]. Rotterdam. |
| 67. | van den Bosch S., Rotmans J. (2008). Deepening, broadening and scaling up: A framework for steering transition experiments. Knowledge Centre for Sustainable System Innovations and Transitions. |
| 68. | van der Ham A., Shields L., Broerse J. (2013). Towards integration of service user knowledge in mental healthcare in low income countries: insights from transition theory. Knowledge Management for Development Journal, 9(2), 125-139. |
| 69. | van Raak R. (2010). The transition (management) perspective on long-term change in healthcare. In: Broerse J.E.W., Bunders J.F.G. (Eds.), Transitions in health systems: Dealing with persistent problems (pp. 49-84). VU University Press. |
| 70. | van Raak R., de Haan A. (2017). Key features of the modern health system: Nature and historical evolution. In Broerse J.E.W., Grin J. (Eds.), Toward sustainable transitions in healthcare systems (pp. 46-84). Routledge. |
| 71. | Ventevogel P., Perez-Sales P., Fernandez-Liria A., Baingana F. (2011). Integrating mental health care into existing systems of health care: During and after complex humanitarian emergencies. Intervention, 9(3), 195-210. |
| 72. | Wakida E., Talib Z. M., Akena D., Okello E. S., Kinengyere A., Mindra A., Obua C. (2018). Barriers and facilitators to the integration of mental health services into primary health care: A systematic review. Systematic Reviews, 7, 211. |
| 73. | |
| 74. | |
| 75. | |
| 76. | WHO. (2019). Scalable psychological interventions for people in communities affected by adversity: A new area of mental health and psychosocial work at WHO. World Health Organization. https://apps.who.int/iris/handle/10665/254581 |
| 77. | |
| 78. | |
| 79. | Yamey G. (2011). Scaling up global health interventions: A proposed framework for success. PLoS Medicine, 8(6), e1001049. |
| 80. | Zamboni K., Schellenberg J., Hanson C., Betran A. P., Dumont A. (2019). Assessing scalability of an intervention: Why, how and who? Health Policy and Planning, 34, 544-552. https://doi.org/10.1093/heapol/czz068 |
[Figure 1]
[Table 1], [Table 2]
|
Search Pubmed for