Annual Research Review
Annual Research Review: The state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field
Nathaniel J. Williams,
Rinad S. Beidas,
Nathaniel J. Williams
School of Social Work, Boise State University, Boise, ID, USA
Both authors contributed equally.Search for more papers by this authorCorresponding Author
Rinad S. Beidas
Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA
Both authors contributed equally.Correspondence
Rinad S. Beidas, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA 19104, USA; Email: [email protected]
Search for more papers by this authorNathaniel J. Williams,
Rinad S. Beidas,
Nathaniel J. Williams
School of Social Work, Boise State University, Boise, ID, USA
Both authors contributed equally.Search for more papers by this authorCorresponding Author
Rinad S. Beidas
Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA
Both authors contributed equally.Correspondence
Rinad S. Beidas, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 3535 Market Street, 3015, Philadelphia, PA 19104, USA; Email: [email protected]
Search for more papers by this authorConflict of interest statement: No conflicts declared.
Abstract
Background
Scientists have developed evidence-based interventions that improve the symptoms and functioning of youth with psychiatric disorders; however, these interventions are rarely used in community settings. Eliminating this research-to-practice gap is the purview of implementation science, the discipline devoted to the study of methods to promote the use of evidence-based practices in routine care.
Methods
We review studies that have tested factors associated with implementation in child psychology and psychiatry, explore applications of social science theories to implementation, and conclude with recommendations to advance implementation science through the development and testing of novel, multilevel, causal theories.
Results
During its brief history, implementation science in child psychology and psychiatry has documented the implementation gap in routine care, tested training approaches and found them to be insufficient for behavior change, explored the relationships between variables and implementation outcomes, and initiated randomized controlled trials to test implementation strategies. This research has identified targets related to implementation (e.g., clinician motivation, organizational culture) and demonstrated the feasibility of activating these targets through implementation strategies. However, the dominant methodological approach has been atheoretical and predictive, relying heavily on a set of variables from heuristic frameworks.
Conclusions
Optimizing the implementation of effective treatments in community care for youth with psychiatric disorders is a defining challenge of our time. This review proposes a new direction focused on developing and testing integrated causal theories. We recommend implementation scientists: (a) move from observational studies of implementation barriers and facilitators to trials that include causal theory; (b) identify a core set of implementation determinants; (c) conduct trials of implementation strategies with clear targets, mechanisms, and outcomes; (d) ensure that behaviors that are core to EBPs are clearly defined; and (e) agree upon standard measures. This agenda will help fulfill the promise of evidence-based practice for improving youth behavioral health.
References
- Aarons, G.A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6, 61–74.
- Aarons, G.A. (2006). Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services, 57, 1162–1169.
- Aarons, G.A., Cafri, G., Lugo, L., & Sawitzky, A. (2012). Expanding the domains of attitudes towards evidence-based practice: The Evidence Based Practice Attitude Scale-50. Administration and Policy in Mental Health and Mental Health Services Research, 39, 331–340.
- Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9, 45.
- Aarons, G.A., Glisson, C., Hoagwood, K.E., Kelleher, K., Landsverk, J., & Cafri, G. (2010). Psychometric properties and US national norms of the Evidence-Based Practice Attitude Scale (EBPAS). Psychological Assessment, 22, 356.
- Aarons, G.A., Hurlburt, M., & Horwitz, S.M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23.
- Aarons, G.A., & Sommerfeld, D.H. (2012). Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 423–431.
- Ajzen, I. (1988). Attitudes, personality, and behavior (p. 166). Dorsey Press.
- Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179–211.
- Albarracin, D., Johnson, B.T., Fishbein, M., & Muellerleile, P.A. (2001). Theories of reasoned action and planned behavior as models of condom use: A meta-analysis. Psychological Bulletin, 127–161, 142.
- Allen, A.P., Kennedy, P.J., Cryan, J.F., Dinan, T.G., & Clarke, G. (2014). Biological and psychological markers of stress in humans: Focus on the Trier Social Stress Test. Neuroscience and Biobehavioral Reviews, 38, 94–124.
- Allen, J.D., Towne, S.D., Maxwell, A.E., DiMartino, L., Leyva, B., Bowen, D.J., … & Weiner, B.J. (2017). Measures of organizational characteristics associated with adoption and/or implementation of innovations: A systematic review. BMC Health Services Research, 17, 591.
- APA Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285.
- Armitage, C.J., & Conner, M. (2001). Efficacy of the theory of planned behaviour: A meta-analytic review. British Journal of Social Psychology, 40, 471–499.
- Arnold, D.R. (2017). Countervailing incentives in value-based payment. Healthcare, 5, 125–128.
- Asgary-Eden, V., & Lee, C.M. (2012). Implementing an evidence-based parenting program in community agencies: What helps and what gets in the way? Administration and Policy in Mental Health and Mental Health Services Research, 39, 478–488.
- Avolio, B.J., Bass, B.M., & Jung, D.I. (1999). Re-examining the components of transformational and transactional leadership using the Multifactor Leadership Questionnaire. Journal of Occupational and Organizational Psychology, 72, 441–462.
- Balas, E.A., & Boren, S.A. (2000). Managing clinical knowledge for health care improvement. Yearbook of medical informatics 2000: Patient-centered systems, 65–70.
- Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215.
- Bass, B.M. (1997). Does the transactional–transformational leadership paradigm transcend organizational and national boundaries? American Psychologist, 52, 130–139.
- Bass, B.M., Avolio, B.J., Jung, D.I., & Berson, Y. (2003). Predicting unit performance by assessing transformational and transactional leadership. Journal of Applied Psychology, 88, 207–218.
- Becker-Haimes, E.M., Okamura, K., Wolk, C.B., Rubin, R., Evans, A.C., & Beidas, R.S. (2017). Predictors of clinician use of exposure therapy in community mental health settings. Journal of Anxiety Disorders, 49, 88–94.
- Beidas, R.S., Edmunds, J.M., Ditty, M., Watkins, J., Walsh, L.M., Marcus, S.C., & Kendall, P. (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy in Mental Health and Mental Health Services Research, 41, 788–799.
- Beidas, R.S., Edmunds, J.M., Marcus, S.C., & Kendall, P.C. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63, 660–665.
- Beidas, R.S., & Kendall, P.C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17, 1–30.
- Beidas, R.S., & Kendall, P.C. (2014). Dissemination and implementation of evidence-based practices in child and adolescent mental health. New York: Oxford University Press.
- Beidas, R.S., Marcus, S.C., Aarons, G.A., Hoagwood, K.E., Schoenwald, S.K., Evans, A.C., … & Mandell, D.S. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169, 374–382.
- Beidas, R.S., Skriner, L., Adams, D., Wolk, C.B., Stewart, R.E., Becker-Haimes, E., … & Weaver, S. (2017). The relationship between consumer, clinician, and organizational characteristics and use of evidence-based and non-evidence-based therapy strategies in a public mental health system. Behaviour Research and Therapy, 99, 1–10.
- Beidas, R.S., Stewart, R.E., Adams, D.R., Fernandez, T., Lustbader, S., Powell, B.J., … & Hurford, M.O. (2016). A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 43, 893–908.
- Berkel, C., Mauricio, A.M., Sandler, I.N., Wolchik, S.A., Gallo, C.G., & Brown, C.H. (2017). The cascading effects of multiple dimensions of implementation on program outcomes: A test of a theoretical model. Prevention Science, 19, 782–794.
- Blackman, D.E. (1974). Operant conditioning: An experimental analysis of behavior. London: Methuen.
- Bosch, M., Van Der Weijden, T., Wensing, M., & Grol, R. (2007). Tailoring quality improvement interventions to identified barriers: A multiple case analysis. Journal of Evaluation in Clinical Practice, 13, 161–168.
- Brimhall, K.C., Fenwick, K., Farahnak, L.R., Hurlburt, M.S., Roesch, S.C., & Aarons, G.A. (2016). Leadership, organizational climate, and perceived burden of evidence-based practice in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 43, 629–639.
- Brookman-Frazee, L., Haine, R.A., Baker-Ericzén, M., Zoffness, R.J., & Garland, A.F. (2010). Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research, 37, 254–269.
- Burgess, A.M., Chang, J., Nakamura, B.J., Izmirian, S., & Okamura, K.H. (2017). Evidence-based practice implementation within a theory of planned behavior framework. Journal of Behavioral Health Services and Research, 44, 647–665.
- Cameron, K.S., & Quinn, R.E. (2011). Diagnosing and changing organizational culture: Based on the competing values framework. San Francisco: John Wiley & Sons.
- Casper, E.S. (2007). The theory of planned behavior applied to continuing education for mental health professionals. Psychiatric Services, 58, 1324–1329.
- Chamberlain, P., Brown, C.H., & Saldana, L. (2011). Observational measure of implementation progress in community based settings: The Stages of Implementation Completion (SIC). Implementation Science, 6, 116.
- Chambers, D.A., Feero, W.G., & Khoury, M.J. (2016). Convergence of implementation science, precision medicine, and the learning health care system: A new model for biomedical research. Journal of the American Medical Association, 315, 1941–1942.
- Chambless, D.L., & Hollon, S.D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7–18.
- Chambless, D.L., & Ollendick, T.H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685–716.
- Chorpita, B.F., Becker, K.D., Daleiden, E.L., & Hamilton, J.D. (2007). Understanding the common elements of evidence-based practice: Misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 647–652.
- Chorpita, B.F., Daleiden, E.L., Ebesutani, C., Young, J., Becker, K.D., Nakamura, B.J., … & Trent, L. (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154–172.
- Collins, P.Y., Patel, V., Joestl, S.S., March, D., Insel, T.R., Daar, A.S., … & Fairburn, C. (2011). Grand challenges in global mental health. Nature, 475, 27–30.
- Committee on Developing Evidence-Based Standards for Psychosocial Interventions for Mental Disorders (2015). Quality measurement. In M.J. England, A.S. Butler & M.L. Gonzalez (Eds.), Psychosocial interventions for mental and substance use disorders: A framework for establishing evidence-based standards (pp. 1–20). Washington, DC: National Academies Press.
- Cooke, R.A., & Rousseau, D.M. (1988). Behavioral norms and expectations: A quantitative approach to the assessment of organizational culture. Group and Organization Studies, 13, 245–273.
- Damschroder, L.J., Aron, D.C., Keith, R.E., Kirsh, S.R., Alexander, J.A., & Lowery, J.C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50.
- Dawson, L., Mullan, B., & Sainsbury, K. (2015). Using the theory of planned behaviour to measure motivation for recovery in anorexia nervosa. Appetite, 84, 309–315.
- Deci, E.L., Koestner, R., & Ryan, R.M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627–668.
- Deci, E.L., & Ryan, R.M. (2002). Overview of self-determination theory: An organismic dialectical perspective. In E.L. Deci & R.M. Ryan (Eds.), Handbook of self-determination research (pp. 3–33). Rochester, NY: The University of Rochester Press.
- Eccles, M.P., Grimshaw, J.M., MacLennan, G., Bonetti, D., Glidewell, L., Pitts, N.B., … & Johnston, M. (2012). Explaining clinical behaviors using multiple theoretical models. Implementation Science, 7, 99.
- Eccles, M.P., Hrisos, S., Francis, J., Kaner, E.F., Dickinson, H.O., Beyer, F., & Johnston, M. (2006). Do self-reported intentions predict clinicians’ behaviour: A systematic review. Implementation Science, 1, 28.
- Eccles, M.P., & Mittman, B.S. (2006). Welcome to Implementation Science. Implementation Science, 1, 1.
- Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44, 350–383.
- Edmunds, J.M., Kendall, P.C., Ringle, V.A., Read, K.L., Brodman, D.M., Pimentel, S.S., & Beidas, R.S. (2013). An examination of behavioral rehearsal during consultation as a predictor of training outcomes. Administration and Policy in Mental Health and Mental Health Services Research, 40, 456–466.
- Ehrhart, M.G., Aarons, G.A., & Farahnak, L.R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157.
- Ehrhart, M.G., Schneider, B., & Macey, W.H. (2013). Organizational climate and culture: An introduction to theory, research, and practice. New York: Routledge.
10.4324/9781315857664 Google Scholar
- Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned action approach. New York: Psychology Press.
- Fishman, J., Beidas, R.S., Blanch, E., & Mandell, D.S. (2018). The utility of measuring intentions to use best practices: A longitudinal study among teachers supporting students with autism. Journal of School Health, 88, 388–395.
- Flottorp, S.A., Oxman, A.D., Krause, J., Musila, N.R., Wensing, M., Godycki-Cwirko, M., … & Eccles, M.P. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implementation Science, 8, 35.
- Garland, A.F., Brookman-Frazee, L., Hurlburt, M.S., Accurso, E.C., Zoffness, R.J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61, 788–795.
- Garland, A.F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health and Mental Health Services Research, 40, 6–22.
- Garner, B.R., Godley, S.H., & Bair, C.M. (2011). The impact of pay-for-performance on therapists’ intentions to deliver high-quality treatment. Journal of Substance Abuse Treatment, 41, 97–103.
- Glisson, C. (2002). The organizational context of children's mental health services. Clinical Child and Family Psychology Review, 5, 233–253.
- Glisson, C., Hemmelgarn, A., Green, P., Dukes, D., Atkinson, S., & Williams, N.J. (2012). Randomized trial of the Availability, Responsiveness, and Continuity (ARC) organizational intervention with community-based mental health programs and clinicians serving youth. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 780–787.
- Glisson, C., Hemmelgarn, A., Green, P., & Williams, N.J. (2013). Randomized trial of the Availability, Responsiveness and Continuity (ARC) organizational intervention for improving youth outcomes in community mental health programs. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 493–500.
- Glisson, C., Landsverk, J., Schoenwald, S.K., Kelleher, K., Hoagwood, K.E., Mayberg, S., … & Research Network on Youth Mental Health (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research, 35, 98.
- Glisson, C., Schoenwald, S.K., Kelleher, K., Landsverk, J., Hoagwood, K.E., Mayberg, S., … & Research Network on Youth Mental Health (2008). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research, 35, 124–133.
- Glisson, C., & Williams, N.J. (2015). Assessing and changing organizational social contexts for effective mental health services. Annual Review of Public Health, 36, 507–523.
- Glisson, C., Williams, N.J., Hemmelgarn, A., Proctor, E.K., & Green, P. (2016). Increasing clinicians’ EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC. Behaviour Research and Therapy, 76, 40–46.
- Godin, G., Bélanger-Gravel, A., Eccles, M., & Grimshaw, J. (2008). Healthcare professionals’ intentions and behaviours: A systematic review of studies based on social cognitive theories. Implementation Science, 3, 36.
- Greenberg, J. (1990). Organizational justice: Yesterday, today, and tomorrow. Journal of Management, 16, 399–432.
- Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581–629.
- Grol, R.P., Bosch, M.C., Hulscher, M.E., Eccles, M.P., & Wensing, M. (2007). Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Quarterly, 85, 93–138.
- Guerrero, E.G., Fenwick, K., & Kong, Y. (2017). Advancing theory development: Exploring the leadership–climate relationship as a mechanism of the implementation of cultural competence. Implementation Science, 12, 133.
- Hartnell, C.A., Ou, A.Y., & Kinicki, A. (2011). Organizational culture and organizational effectiveness: A meta-analytic investigation of the competing values framework's theoretical suppositions. Journal of Applied Psychology, 96, 677.
- Henderson, C.E., Young, D.W., Jainchill, N., Hawke, J., Farkas, S., & Davis, M.R. (2007). Program use of effective drug abuse treatment practices for juvenile offenders. Journal of Substance Abuse Treatment, 32, 279–290.
- Henggeler, S.W., Chapman, J.E., Rowland, M.D., Halliday-Boykins, C.A., Randall, J., Shackelford, J., & Schoenwald, S.K. (2008). Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. Journal of Consulting and Clinical Psychology, 76, 556–567.
- Herschell, A.D., Kolko, D.J., Baumann, B.L., & Davis, A.C. (2010). The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical Psychology Review, 30, 448–466.
- Higa-McMillan, C.K., Nakamura, B.J., Morris, A., Jackson, D.S., & Slavin, L. (2015). Predictors of use of evidence-based practices for children and adolescents in usual care. Administration and Policy in Mental Health and Mental Health Services Research, 42, 373–383.
- Howarth, E., Devers, K., Moore, G., O'Cathain, A., & Dixon-Woods, M. (2016). Contextual issues and qualitative research. In R. Raine, R. Fitzpatrick, H. Barratt, G. Bevan, N. Black, R. Boaden, P. Bower, M. Campbell, J.-L. Denis & K. Devers (Eds.), Challenges, solutions and future directions in the evaluation of service innovations in health care and public health, Vol. 4. Southampton, UK: NIHR Journals Library.
- Insel, T.R. (2013). NIMH's new focus in clinical trials. Available from: https://www.nimh.nih.gov/funding/grant-writing-and-application-process/concept-clearances/2013/nimhs-new-focus-in-clinical-trials.shtml [last accessed 01 February 2018].
- Jacobs, S.R., Weiner, B.J., & Bunger, A.C. (2014). Context matters: Measuring implementation climate among individuals and groups. Implementation Science, 9, 46.
- James, L.R., Choi, C.C., Ko, C.E., McNeil, P.K., Minton, M.K., Wright, M.A., & Kim, K. (2008). Organizational and psychological climate: A review of theory and research. European Journal of Work and Organizational Psychology, 17, 5–32.
- Jensen-Doss, A., Hawley, K.M., Lopez, M., & Osterberg, L.D. (2009). Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice, 40, 417.
- Judge, T.A., & Piccolo, R.F. (2004). Transformational and transactional leadership: A meta-analytic test of their relative validity. Journal of Applied Psychology, 89, 755–768.
- Judson, T.J., Volpp, K.G., & Detsky, A.S. (2015). Harnessing the right combination of extrinsic and intrinsic motivation to change physician behavior. Journal of the American Medical Association, 314, 2233–2234.
- Kazdin, A.E. (2018). Innovations in psychosocial interventions and their delivery: Leveraging cutting-edge science to improve the world's mental health. New York: Oxford University Press.
10.1093/med-psych/9780190463281.001.0001 Google Scholar
- Klein, K.J., Conn, A.B., & Sorra, J.S. (2001). Implementing computerized technology: An organizational analysis. Journal of Applied Psychology, 86, 811–824.
- Klein, K.J., Dansereau, F., & Hall, R.J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195–229.
- Klein, K.J., & Sorra, J.S. (1996). The challenge of innovation implementation. Academy of Management Review, 21, 1055–1080.
- Koolhaas, J., Bartolomucci, A., Buwalda, B., De Boer, S., Flügge, G., Korte, S., … & Palanza, P. (2011). Stress revisited: A critical evaluation of the stress concept. Neuroscience and Biobehavioral Reviews, 35, 1291–1301.
- Lehman, W.E., Greener, J.M., & Simpson, D.D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209.
- Lewis, C.C., Fischer, S., Weiner, B.J., Stanick, C., Kim, M., & Martinez, R.G. (2015). Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science, 10, 155.
- Lim, A., Nakamura, B.J., Higa-McMillan, C.K., Shimabukuro, S., & Slavin, L. (2012). Effects of workshop trainings on evidence-based practice knowledge and attitudes among youth community mental health providers. Behaviour Research and Therapy, 50, 397–406.
- Locke, J., Wolk, C.B., Harker, C., Olsen, A., Shingledecker, T., Barg, F., … & Beidas, R.S. (2017). Pebbles, rocks, and boulders: The implementation of a school-based social engagement intervention for children with autism. Autism, 21, 985–994.
- Lomas, J. (1993). Diffusion, dissemination, and implementation: Who should do what? Annals of the New York Academy of Sciences, 703, 226–237.
- Lynch, M.F., Jr, Plant, R.W., & Ryan, R.M. (2005). Psychological needs and threat to safety: Implications for staff and patients in a psychiatric hospital for youth. Professional Psychology: Research and Practice, 36, 415–425.
- MacWilliams, K., Curran, J., Racek, J., Cloutier, P., & Cappelli, M. (2017). Barriers and facilitators to implementing the HEADS-ED: A rapid screening tool for pediatric patients in emergency departments. Pediatric Emergency Care, 33, 774–780.
- Maslach, C., Jackson, S.E., Leiter, M.P., Schaufeli, W.B., & Schwab, R.L. (1986). Maslach Burnout Inventory, Vol. 21. Palo Alto, CA: Consulting Psychologists Press.
- Mathieu, J.E., & Zajac, D.M. (1990). A review and meta-analysis of the antecedents, correlates, and consequences of organizational commitment. Psychological Bulletin, 108, 171.
- McHugh, R.K., & Barlow, D.H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65, 73.
- Michie, S., Johnston, M., Abraham, C., Lawton, R., Parker, D., & Walker, A. (2005). Making psychological theory useful for implementing evidence based practice: A consensus approach. BMJ Quality and Safety, 14, 26–33.
- Miklowitz, D.J., Goodwin, G.M., Bauer, M.S., & Geddes, J.R. (2008). Common and specific elements of psychosocial treatments for bipolar disorder: A survey of clinicians participating in randomized trials. Journal of Psychiatric Practice, 14, 77.
- Miller, D.G., & Sittig, D.F. (2017). Underperforming big ideas in biomedical research. Journal of the American Medical Association, 317, 321–322.
- Miller, W.R., Yahne, C.E., Moyers, T.B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050–1062.
- Nakamura, B.J., Higa-McMillan, C.K., Okamura, K.H., & Shimabukuro, S. (2011). Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Administration and Policy in Mental Health and Mental Health Services Research, 38, 287–300.
- Ng, M.Y., & Weisz, J.R. (2016). Annual research review: Building a science of personalized intervention for youth mental health. Journal of Child Psychology and Psychiatry, 57, 216–236.
- Okamura, K.H., Nakamura, B.J., Mueller, C., Hayashi, K., & Higa-McMillan, C.K. (2016). An exploratory factor analysis of the knowledge of evidence-based services questionnaire. Journal of Behavioral Health Services and Research, 43, 214–232.
- Palinkas, L.A., Um, M.Y., Jeong, C.H., Chor, K.H.B., Olin, S., Horwitz, S.M., & Hoagwood, K.E. (2017). Adoption of innovative and evidence-based practices for children and adolescents in state-supported mental health clinics: A qualitative study. Health Research Policy and Systems, 15, 27.
- Papoutsi, C., Boaden, R., Foy, R., Grimshaw, J., & Rycroft-Malone, J. (2016). Challenges for implementation science. In R. Raine, R. Fitzpatrick, H. Barratt, G. Bevan, N. Black, R. Boaden, P. Bower, M. Campbell, J.-L. Denis & K. Devers (Eds.), Challenges, solutions and future directions in the evaluation of service innovations in health care and public health, Vol. 4. Southampton, UK: NIHR Journals Library.
- Potthoff, S., Presseau, J., Sniehotta, F.F., Johnston, M., Elovainio, M., & Avery, L. (2017). Planning to be routine: Habit as a mediator of the planning-behaviour relationship in healthcare professionals. Implementation Science, 12, 24.
- Powell, B.J., Beidas, R.S., Lewis, C.C., Aarons, G.A., McMillen, J.C., Proctor, E.K., & Mandell, D.S. (2017). Methods to improve the selection and tailoring of implementation strategies. Journal of Behavioral Health Services and Research, 44, 177–194.
- Powell, B.J., Mandell, D.S., Hadley, T.R., Rubin, R.M., Evans, A.C., Hurford, M.O., & Beidas, R.S. (2017). Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study Implementation Science, 12, 64.
- Powell, B.J., Waltz, T.J., Chinman, M.J., Damschroder, L.J., Smith, J.L., Matthieu, M.M., … & Kirchner, J.E. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 21.
- Presseau, J., Johnston, M., Heponiemi, T., Elovainio, M., Francis, J.J., Eccles, M.P., … & Grimshaw, J.M. (2014). Reflective and automatic processes in health care professional behaviour: A dual process model tested across multiple behaviours. Annals of Behavioral Medicine, 48, 347–358.
- Prochaska, J.O., & Velicer, W.F. (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12, 38–48.
- Proctor, E.K., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G.A., Bunger, A., … & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76.
- Rogers, E.M. (2003). Diffusion of innovations ( 5th edn). New York: Simon and Schuster.
- Rohrbach, L.A., Graham, J.W., & Hansen, W.B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260.
- Russell, E., Koren, G., Rieder, M., & Van Uum, S. (2012). Hair cortisol as a biological marker of chronic stress: Current status, future directions and unanswered questions. Psychoneuroendocrinology, 37, 589–601.
- Ryan, R.M., & Deci, E.L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68.
- Ryan, R.M., & Deci, E.L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. New York: Guilford.
10.1521/978.14625/28806 Google Scholar
- Saldana, L., Chamberlain, P., Wang, W., & Brown, C.H. (2012). Predicting program start-up using the stages of implementation measure. Administration and Policy in Mental Health and Mental Health Services Research, 39, 419–425.
- Saldana, L., Chapman, J.E., Henggeler, S.W., & Rowland, M.D. (2007). The Organizational Readiness for Change scale in adolescent programs: Criterion validity. Journal of Substance Abuse Treatment, 33, 159–169.
- Schein, E.H. (2010). Organizational culture and leadership, Vol. 2. San Francisco: John Wiley & Sons.
- Schiele, B.E. (2013). The importance of counseling self-efficacy in school mental health. Columbia, SC: University of South Carolina.
- Schneider, B., Ehrhart, M.G., & Macey, W.H. (2013). Organizational climate and culture. Annual Review of Psychology, 64, 361–388.
- Schoenwald, S.K. (2011). It's a bird, it's a plane, it's… fidelity measurement in the real world. Clinical Psychology: Science and Practice, 18, 142–147.
- Schoenwald, S.K., Carter, R.E., Chapman, J.E., & Sheidow, A.J. (2008). Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Administration and Policy in Mental Health and Mental Health Services Research, 35, 379–394.
- Sheeran, P. (2002). Intention—behavior relations: A conceptual and empirical review. European Review of Social Psychology, 12, 1–36.
10.1080/14792772143000003 Google Scholar
- Sheppard, B.H., Hartwick, J., & Warshaw, P.R. (1988). The theory of reasoned action: A meta-analysis of past research with recommendations for modifications and future research. Journal of Consumer Research, 15, 325–343.
- Sholomskas, D.E., Syracuse-Siewert, G., Rounsaville, B.J., Ball, S.A., Nuro, K.F., & Carroll, K.M. (2005). We don't train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73, 106–115.
- Smith, G.P., & Williams, T.M. (2017). You can lead a horse to water… what self-determination theory can contribute to our understanding of clinical policy implementation. Journal of Health Services Research and Policy, 22, 37–44.
- Stein, B.D., Celedonia, K.L., Kogan, J.N., Swartz, H.A., & Frank, E. (2013). Facilitators and barriers associated with implementation of evidence-based psychotherapy in community settings. Psychiatric Services, 64, 1263–1266.
- Stephan, S., Westin, A., Lever, N., Medoff, D., Youngstrom, E., & Weist, M. (2012). Do school-based clinicians’ knowledge and use of common elements correlate with better treatment quality? School Mental Health, 4, 170–180.
- Stumpf, R.E., Higa-McMillan, C.K., & Chorpita, B.F. (2009). Implementation of evidence-based services for youth: Assessing provider knowledge. Behavior Modification, 33, 48–65.
- Tabak, R.G., Khoong, E.C., Chambers, D.A., & Brownson, R.C. (2012). Bridging research and practice: Models for dissemination and implementation research. American Journal of Preventive Medicine, 43, 337–350.
- Turner, S., Goulding, L., Denis, J.-L., McDonald, R., & Fulop, N.J. (2016). Major system change: A management and organisational research perspective. In R. Raine, R. Fitzpatrick, H. Barratt, G. Bevan, N. Black, R. Boaden, P. Bower, M. Campbell, J.-L. Denis & K. Devers (Eds.), Challenges, solutions and future directions in the evaluation of service innovations in health care and public health (Vol. 4; pp. 85–104). Southampton, UK: NIHR Journals Library.
- Wang, W., Saldana, L., Brown, C.H., & Chamberlain, P. (2010). Factors that influenced county system leaders to implement an evidence-based program: A baseline survey within a randomized controlled trial. Implementation Science, 5, 72.
- Weersing, V.R., Rozenman, M., & Gonzalez, A. (2009). Core components of therapy in youth: Do we know what to disseminate? Behavior Modification, 33, 24–47.
- Weiner, B.J., Amick, H., & Lee, S.D. (2008). Conceptualization and measurement of organizational readiness for change: A review of the literature in health services research and other fields. Medical Care Research and Review, 65, 379–436.
- Weiner, B.J., Belden, C.M., Bergmire, D.M., & Johnston, M. (2011). The meaning and measurement of implementation climate. Implementation Science, 6, 78.
- Weiner, B.J., Lewis, C.C., Stanick, C., Powell, B.J., Dorsey, C.N., Clary, A.S., … & Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 108.
- Weisz, J.R., Donenberg, G.R., Han, S.S., & Kauneckis, D. (1995). Child and adolescent psychotherapy outcomes in experiments versus clinics: Why the disparity? Journal of Abnormal Child Psychology, 23, 83–106.
- Weisz, J.R., Donenberg, G.R., Han, S.S., & Weiss, B. (1995). Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63, 688–701.
- Weisz, J.R., Jensen-Doss, A., & Hawley, K.M. (2005). Youth psychotherapy outcome research: A review and critique of the evidence base. Annual Review of Psychology, 56, 337–363.
- Weisz, J.R., Kuppens, S., Eckshtain, D., Ugueto, A.M., Hawley, K.M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761.
- Weisz, J.R., Ng, M.Y., & Bearman, S.K. (2014). Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era. Clinical Psychological Science, 2, 58–74.
10.1177/2167702613501307 Google Scholar
- Weisz, J.R., Ugueto, A.M., Cheron, D.M., & Herren, J. (2013). Evidence-based youth psychotherapy in the mental health ecosystem. Journal of Clinical Child and Adolescent Psychology, 42, 274–286.
- Williams, N.J. (2015). Assessing mental health clinicians’ intentions to adopt evidence-based treatments: Reliability and validity testing of the evidence-based treatment intentions scale. Implementation Science, 11, 60.
- Williams, N.J. (2016). Multilevel mechanisms of implementation strategies in mental health: Integrating theory, research, and practice. Administration and Policy in Mental Health and Mental Health Services Research, 43, 783–798.
- Williams, V., Deane, F.P., Oades, L.G., Crowe, T.P., Ciarrochi, J., & Andresen, R. (2015). A cluster-randomised controlled trial of values-based training to promote autonomously held recovery values in mental health workers. Implementation Science, 11, 13.
- Williams, N.J., Ehrhart, M.G., Aarons, G.A., Marcus, S.C., & Beidas, R.S. (2018). Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: Cross-sectional and lagged results from a two-year observational study. Implementation Science.
- Williams, N.J., & Glisson, C. (2014). The role of organizational culture and climate in the dissemination and implementation of empirically-supported treatments for youth. In R.S. Beidas & P.C. Kendall (Eds.), Dissemination and implementation of evidence based practices in child and adolescent mental health (pp. 61–81). New York: Oxford University Press.
- Williams, N.J., & Glisson, C. (2018). Changing organizational social context to support evidence-based practice implementation: A conceptual and empirical review. In R. Mildon, B. Albers & A. Shlonsky (Eds.), The science of implementation. New York: Springer.
- Williams, N.J., Glisson, C., Hemmelgarn, A., & Green, P. (2017). Mechanisms of change in the ARC organizational strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration and Policy in Mental Health and Mental Health Services Research, 44, 269–283.
- Williams, J.R., Williams, W.O., Dusablon, T., Blais, M.P., Tregear, S.J., & Banks, D. (2014). Evaluation of a randomized intervention to increase adoption of comparative effectiveness research by community health organizations. Journal of Behavioral Health Services and Research, 41, 308–323.
- Wolk, C.B., Jager-Hyman, S., Marcus, S.C., Ahmedani, B.K., Zeber, J.E., Fein, J.A., … & Beidas, R.S. (2017). Developing implementation strategies for firearm safety promotion in paediatric primary care for suicide prevention in two large US health systems: A study protocol for a mixed-methods implementation study. British Medical Journal Open, 7, e014407.
- Zazzali, J.L., Sherbourne, C., Hoagwood, K.E., Greene, D., Bigley, M.F., & Sexton, T.L. (2008). The adoption and implementation of an evidence based practice in child and family mental health services organizations: A pilot study of functional family therapy in New York State. Administration and Policy in Mental Health and Mental Health Services Research, 35, 38–49.
- Zohar, D., & Hofmann, D.A. (2012). Organizational culture and climate. Oxford Handbook of Industrial and Organizational Psychology, 1, 643–666.
- Zohar, D., & Polachek, T. (2014). Discourse-based intervention for modifying supervisory communication as leverage for safety climate and performance improvement: A randomized field study. Journal of Applied Psychology, 99, 113–124.
