Fluent Essays

Respond to at least two colleagues by  offering critiques of their analyses. Identify strengths in their analyses and strategies for presenting evaluation results to others.  dentify ways your colleagues might improve their presentations.  Identify potential needs or questions of the audience that they may not have considered.  Provide an additional strategy for overcoming the obstacles or challenges in communicating the content of the evaluation reports. Presentation to Colleagues Research is key to social work practice and helps validate the data behind a successful program. Using a process evaluation is unique in the way that it is used to evaluate a program that is actively operating to address gaps, areas for improvement, and areas of success (Bliss & Emshoff, 2002). The important distinctions of process evaluation must be well described in order to not confuse it with outcome evaluations. Rather than address solely the outcome of a program, process evaluations aim to address how the program is or is not effective and how it may be improved or implemented differently (Law & Shek, 2011). For this case example, teaching about this difference can impact the results of the research to ensure that the process of the intervention is assessed rather than the final results. For the case example, the PATHS program obtained data through surveys based on observations made by independent researchers (Law & Shek, 2011). This design supports the concept of process evaluation to ensure that it focuses on the program implementation and the program success (Law & Shek, 2011). Additionally, having independent observers supports more unbiased information and assessment. When delivering the results of the PATHS program, it’s important to address the specific research questions and also present data. Colleagues closely involved with the program may be concerned that the evaluation was inaccurate or does not adequately portray the abilities of the program, so presenting data can support the results. Overall, the importance of designing an adequate research method for a program is essential to addressing an issue and answering specific research questions. Background and Key Message Training and ongoing education is essential to any profession including social work. Ensuring that research and evaluation models are not only taught, but understood is an important element to social work practice and promoting evidence based treatment. In this case, the importance of addressing an appropriate model for research can support valuable results. For instance, designing specific and broad research questions may assist in developing the type of model that may be most effective. Educating colleagues on what these research questions were would be helpful. Determining whether the model is collecting qualitative or quantitative data may be instrumental depending on the questions being asked. In this case, addressing the success of the intervention may be measured from the observation of researchers. In a different study, the purpose may be to develop an understanding for training staff, so observing or interviewing staff may be more appropriate. The social worker’s ability to determine which research model will support their research and outcomes. When educating colleagues about these differences, it’s important to provide insight to why this model was chosen to evaluate the program so there is understanding of why this model was considered more appropriate than others. Questions and Response In order to educate colleagues, addressing their questions or concerns with the process evaluation may support their learning. While social workers and other professionals should be interested in ongoing education, supporting the education with topics of interest is, of course, a method to support their interest. As references previously, the concern for choosing which model of research may be most beneficial if likely to be a question. Additionally, if results did not prove effective intervention, there may be concern that the evaluation did not adequately portray the importance of the program. Either of these questions or concerns may be expected among colleagues of the program being evaluated. By addressing the questions and supporting the answers with data and research-based information, concern can be minimized. Lastly, colleagues may have questions related to how to best implement research within their scope of practice and may feel intimidated by the information, so ensuring practical skills and direct application or these research models may be insightful as an educator. Overall, the purpose of using research and how to carry-out research is expected to be beneficial for the colleagues and hopefully, would encourage further research in the field. References Bliss, M. J., & Emshoff, J. G. (2002). Workbook for designing a process evaluation. Retrieved from http://beta.roadsafetyevaluation.com/evaluationguides/info/workbook- for-designing-a-process-valuation.pdf Dudley, J. R. (2020). Social work evaluation: Enhancing what we do (3rd ed.) Oxford University Press. Law, B., & Shek, D. (2011). Process Evaluation of a Positive Youth Development Program: Project P.A.T.H.S. Research on Social Work Practice, 21(5), 539-548. Research Article Process Evaluation of a Positive Youth Development Program: Project P.A.T.H.S. Ben M. F. Law 1 and Daniel T. L. Shek 2,3,4 Abstract There are only a few process evaluation studies on positive youth development programs, particularly in the Chinese context. Objectives: This study aims to examine the quality of implementation of a positive youth development program (Project Positive Adolescent Training through Holistic Social Programs [P.A.T.H.S.]) and investigate the relationships among program adherence, process factors, implementation quality, and success. Method: Process evaluation of 20 Secondary 3 classroom- based programs was conducted in 14 schools. Results: Overall program adherence, individual evaluation items, quality, and success had high ratings. Principal components analysis showed that two components, namely, implementation process and implementation context were extracted from 11 evaluation items. The correlational analysis indicated that program adherence, implementation process, and context were highly correlated with quality and success. Multiple regression analyses show that teaching process and program adherence predicted quality, whereas teaching process, teaching context, and program adherence predicted success. Conclusions: The implementation quality of the Tier 1 Program of Project P.A.T.H.S. was generally high. Keywords Project P.A.T.H.S, process evaluation, positive youth development program Introduction Social work programs are specific sets of strategies and actions that can be implemented to enhance social functioning and problem-solving capabilities among individuals, families, and groups. Program evaluation is a systematic assessment of the process and outcomes of the programs with the aim of contri- buting to the improvement of the programs, such as in deciding whether to adopt the program further, enhancement of existing intervention protocols, and compliance with a set of explicit or implicit standards (Zakrzewski, Steven, & Ricketts, 2009). This article documents the process evaluation of a large-scale positive youth development program in Hong Kong called Pos- itive Adolescent Training through Holistic Social Programs (P.A.T.H.S.). Process Evaluation in Prevention Science and Social Work Practice Outcome evaluation focuses mainly on the results of the programs, whereas process evaluation is concerned with how the program is actually delivered (Dane & Schneider, 1998; Domitrovich & Greenberg, 2000). Process evaluation is widely adopted in prevention science, such as nursing care (Huryk, 2010; Painter et al., 2010), chronic illness prevention programs (Braun et al., 2010; Karwalajtys et al., 2009; Mair, Hiscock, & Beaton, 2008; Shevil & Finlayson, 2009), smoking cessation programs (Gnich, Sheehy, Amos, Bitel, & Platt, 2008; Kwong et al., 2009; Quintiliani, Yang, & Sorensen, 2010), dietary programs (Allicock et al., 2010; Bowes, Marquis, Young, Holowaty, & Issac, 2009; Hart et al., 2009; Muckelbuer, Libuda, Clausen, & Kersting, 2009; Salmela, Poskiparta, Kasila, Vahasarja, & Vanhala, 2009), and AIDS rehabilitation programs (Bertens, Eiling, van den Borne, & Schaalma, 2009; Fraze et al., 2009; Hargreaves et al., 2009; Konle-Parker, Erien, & Dubbert, 2010; Mukoma et al., 2009). In social work prac- tice, process evaluation has been used in family programs (Cohen, Glynn, Hamilton, & Young, 2010; Kumpfer, Pinyuchon, de Melo, & Whiteside, 2008) but is not commonly used in youth programs (Beets et al., 2008; Frazen, Morrel-Samuels, 1 Department of Social Work and Social Administration, The University of Hong Kong, Hong Kong 2 Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hong Kong 3 Public Policy Research Institute, The Hong Kong Polytechnic University, Hong Kong 4 College of Medicine, University of Kentucky, KY, USA, Corresponding Author: Ben M. F. Law, Department of Social Work, The Chinese University of Hong Kong, Hong Kong Email: [email protected] Research on Social Work Practice 21(5) 539-548 ª The Author(s) 2011 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/1049731511404436 http://rsw.sagepub.com http://crossmark.crossref.org/dialog/?doi=10.1177\%2F1049731511404436&domain=pdf&date_stamp=2011-03-31 Reischl, & Zimmerman, 2009; Johnson, Lai, Rice, Rose, & Webber, 2010). Process evaluation consists of five components, namely, program adherence, implementation process, intended dosage, macro-level implication, and process-outcome linkage (Scheirer, 1994). Program adherence deals with whether the program is being delivered as intended according to the original program design. It is an important factor affecting the quality of program imple- mentation (Domitrovich & Greenberg, 2000; Fagan, Hanson, Hawkins, & Arthur, 2008). True program fidelity is not easily achieved because program implementers often change or adapt the program content during actual implementation, whether intentionally or otherwise. Studies have shown that a number of preventive programs do not follow the prescribed program content entirely, and adaptation is made to specific target groups (Elliot & Mihalic, 2004; Nation et al., 2003). A study has found tension between the desire of the program implemen- ter to adhere to the manualized plan and to make adaptations in accordance with the needs of clients (Wegner, Flisher, Caldwell, Vergnani, & Smith, 2008). Although it is not an easily resolved issue, program fidelity is generally encouraged, especially when programs are designed with vigorous trial runs and repeated success rates (Griffin et al., 2010; Johnson et al., 2010; Wilson et al., 2009). Process factors are those that can be observed during the implementation process and are contingent to implementation quality or success. There is a variety of process factors accord- ing to the program characteristics and the needs of program developers. Some programs even design their own process measurements (Yamada, Stevens, Sidani, Watt-Watson, & de Silva, 2009). There are two main groups of process indicators in prevention science and social work programs. First is the implementation process. It is the direct observation of the inter- action between the program implementer and the program receivers, such as the program receivers’ engagement and the program implementer’s use of feedback. Second is the imple- mentation context. It involves context factors critical to imple- mentation, including goal attainment and background knowledge, such as the program implementer’s familiarity with the program receivers and the program implementer’s program preparation. Program dosage refers to the effort by program implemen- ters to follow the required time prescribed for a program, as inadequate time affects the quality of program implementation (Bowes et al., 2009; Johnson et al., 2010). Dosage also refers to the group size of program receivers. A discrepancy between the intended and actual program receiver to program implementer ratio affects the program delivery process (Frazen et al., 2009). Process evaluation can provide important findings with macro-level program implications, such as the importance of engagement of different community stakeholders (Carswell, Hanlon, O’Grady, Watts, & Pothong, 2009; Zani & Cicognani, 2010), client needs (Kwong et al., 2009), assessment of the environment (Eisenberg, 2009; Stewart, 2008), and challenges of the programs for a particular context (Louis et al., 2008). Process evaluation and outcome evaluation are strongly linked. Process evaluation sheds light on which types of inter- ventions strategies or process are related to the program success (Kwong et al., 2009; Painter et al., 2010). These factors can be amplified during program reimplementation. The components of process evaluation point toward its importance. First, outcome evaluation provides inadequate hints on the quality of program implementation. Process eva- luation demystifies the ‘‘black box’’ of intervention and aids in the understanding of the elements of program success or fail- ure (Harachi, Abbot, Catalnao, Haggerty, & Fleming, 1999). Process evaluation facilitates program developers to under- stand fully the strengths and weaknesses of the developed pro- grams. Program implementers can follow the suggestions from the process evaluation for further program delivery. This is one essence of evidence-based practice. It is also the foundation of bridging the gap between research and practice (Saul et al., 2008; Wandersman et al., 2008). Second, process evaluation can inform program developers about whether the programs are delivered according to some standardized manuals. The exis- tence of other activities different from those intended by the program developers will not truly reflect the effectiveness of the prescribed programs. Third, different human organizations and communities arrange the programs in various settings, levels of involvement by the stakeholders, perceptions of the program among program implementers and program receivers, as well as the levels of support. Process evaluation can document the variety of implementations in real human service settings for the same manualized plans. Finally, process evaluation provides insights for program developers and implementers into the link- age between process and outcome. These insights allow both program developers and implementers to delineate the success and improvement areas during the process and connect them with the program outcomes. Project P.A.T.H.S. Many primary prevention programs and positive youth devel- opment programs have been developed in the West to address the growing adolescent development problems, such as sub- stance abuse, mental health problems, and school violence (Shek, 2006a; Shek & Merrick, 2009). However, in Hong Kong, there are very few systematic and multiyear positive youth development programs. To promote holistic develop- ment among adolescents in Hong Kong, The Hong Kong Jockey Club Charities Trust approved the release of HK$750 million (HK$400 million for the first phase and HK$350 mil- lion for the second phase) to launch a project entitled ‘‘P.A.T.H.S. to Adulthood: A Jockey Club Youth Enhancement Scheme.’’ The acronym ‘‘P.A.T.H.S.’’ denotes Positive Adoles- cent Training through Holistic Social Programs. The Trust invited academics from five universities in Hong Kong to form a research team to develop a multiyear universal positive youth program (Shek & Merrick, 2009). The project commenced in 2004 and is targeted to end by 2012. There are two tiers of programs in this project. The Tier 540 Research on Social Work Practice 21(5) 1 Program is a universal positive youth development program where students from the Secondary 1 (Grade 7) to Secondary 3 (Grade 9) participate in a classroom-based program, normally with 20 hr of training in the school year in each grade. Around one fifths of adolescents with more psychosocial needs will join the Tier 2 Program. The Tier 2 Program consists of inten- sive training on volunteer service, adventure-based counseling camp, and other experiential learning activities. The overall objective of the Tier 1 Program is to promote holistic development among junior secondary school students in Hong Kong. The programs are designed according to 15 con- structs conducive to adolescent development (Shek, 2006b): promotion of bonding, cultivation of resilience, promotion of social competence, promotion of emotional competence, promotion of cognitive competence, promotion of behavioral competence, promotion of moral competence, cultivation of self-determination, promotion of spirituality, development of self-efficacy, development of a clear and positive identity, promotion of beliefs in the future, provision of recognition for positive behavior, provision of opportunities for prosocial involvement, and promotion of prosocial norms. The Tier 1 Program has several characteristics. First, there are 40 units per grade (each lasting for 30 min), with a total of 120 units, for the entire Tier 1 program. The time fits well with the Hong Kong secondary school time slots. Second, each school can choose to implement all 40 units (full program) or 20 units (core program), according to school needs. Third, the program content was developed by the research team and underwent extensive integration of existing research findings, adolescent needs, cultural characteristics, and trial teaching runs. Fourth, relevant adolescent developmental issues, such as drug issues, sexuality, and financial management, are incor- porated into the program content so that it fits the current real- life experiences of Hong Kong adolescents. Fifth, the program implementers are either social workers or teachers who had to undergo intensive 20-hr training before program delivery. There are two implementation phases: the experimental implementation phase (EIP) and the full implementation phase (FIP). The EIP aims at accumulating experience from trial teaching and administrative arrangement. Program materials are revised and refined during this phase. The FIP aims at executing the programs in full force. There are several lines of evidence that support the effectiveness of the Tier 1 Program, including the evaluation findings based on rando- mized group trials (e.g., Shek & Ma, 2011; Shek & Yu, 2011), subjective outcome evaluation (e.g., Shek & Sun, 2007), quali- tative findings based on focus group interviews with program implementers and students (e.g., Shek & Lee, 2008), interim evaluation (e.g., Shek, Sun, & Siu, 2008), analyses of the weekly diaries of students (e.g., Shek, Sun, Lam, Lung, & Lo, 2008), and case studies (e.g., Shek & Sun, 2008). The evaluation findings based on different evaluation strategies indicate that Project P.A.T.H.S. promotes the development of its program participants. Process evaluation has already been carried out in the EIP and FIP for Secondary 1 (Shek, Ma, Lui, & Lung, 2006) and 2 students (Shek, Lee, & Sun, 2008). The evaluation results indicate that the quality of implementation and program adher- ence are high. The current study focuses on the Secondary 3 Tier 1 Program. Process Evaluation for the Secondary 3 Tier 1 Program Process evaluation for Secondary 3 students is important. First, the Secondary 3 curriculum is different from the others. It requires students to develop self-reflexivity during the process. Thus, the findings of the process evaluation may be different. Second, Secondary 3 students are cognitively more mature and have more life exposure than their Secondary 1 and 2 counterparts. Their perception of program implementation quality can be different. Finally, Secondary 3 students have participated in Project P.A.T.H.S. continuously for 3 years and have all completed the entire Tier 1 curriculum. Their feedback represents an overall evaluation for the entire Project P.A.T.H.S. curriculum. The current process evaluation focuses on program adher- ence, process factors, program quality, and success. Program adherence is the objective estimation of the adoption percentage from the manualized plan for real service delivery. A variety of process factors exist. A review of literature indicates that the following program attributes can affect the quality and success of the positive youth development program implementation (Collaborative for Academic, Social and Emotional Learning, 2010; Harachi et al., 1999; Nation et al., 2003; Ringwalt et al., 2003; Tobler, Lessard, Marshall, Ochshorn, & Roona, 1999): 1. Student interest: A successful program usually elicits the interest of students. 2. Active involvement of students: The more involved the students are, the higher the possibility that the program can achieve its outcomes. 3. Classroom management: The program implementer can manage student discipline during student activities. Stu- dents obey the requirements set by the program imple- menter and are attentive. 4. Interactive delivery method: Interactive delivery is better than didactic delivery for positive youth development programs. 5. Strategies to enhance the motivation of students: The use of various learning strategies can enhance the engage- ment of students and result in positive learning outcomes. 6. Positive feedback: The use of praise and encouragement throughout the lessons by the program implementers can promote the engagement of students. 7. Familiarity of implementers with the students: All other things being equal, a higher degree of familiarity with the students is positively related to student learning outcome. 8. Reflective learning: The program implementer should engage students in reflection and deeper learning. This can lead to growth and meaningful changes among the students. Law and Shek 541 9. Program goal attainment: The achievement of program goals constitutes program success. 10. Time management: Efficient time management ensures that the majority of the program materials are carried out with high program adherence. 11. Familiarity of program implementers with the implementa- tion materials: Familiarity with the material ensures that the messages are conveyed effectively to the students. Program quality is the subjective appraisal of the program implementation process. It can be reflected from the implemen- tation atmosphere and the interaction between program imple- menters and students. Program success refers to the extent of unit objective attainment and the subjective evaluation of the response of the students to the program. Against this background, the current study aims to explore the factors related to the implementation quality and imple- mentation success of the Secondary 3 Tier 1 Program during the Full Implementation Phase. There are two research questions: 1. What is the implementation quality of the Secondary 3 curriculum of the Tier 1 Program of Project P.A.T.H.S. in Hong Kong? 2. How are program adherence and other indicators related to the implementation quality and success of the Secondary 3 Tier 1 Program? Method Participants and Procedure In total, 14 schools were randomly selected from among the 167 secondary schools that joined the Secondary 3 program in the school year 2008/2009 for the process evaluation. Process evaluation was carried out using systematic obser- vations of actual classroom program delivery. For each school joining the process evaluation, one to two program units were evaluated by two independent observers who are project colleagues with master’s degrees. A total of 20 units were observed for this study. The learning units of these units are shown in Table 1. During the observation, observers sat at the back of the classroom and evaluated the method by which the units were actually implemented by completing several instruments. After the psychometric properties of the instruments were explored, program adherence and implementation process components were associated with implementation quality and success. In addition, program and implementation process components were used to predict implementation quality and implementation success separately. Instruments Program adherence. Observers were requested to rate program adherence in terms of percentage (i.e., the correspondence between actual program delivery and stipulated program mate- rials). Pearson correlation analyses showed that the ratings of program adherence were highly reliable (r ¼ .86, p < .001) between raters. Implementation Process Checklist (IPC) The IPC consists of 11 items, which are shown in Table 2. Items 1, 2, 3, 4, 5, 6, and 8 are conceptually related to the imple- mentation process, whereas items 7, 9, 10, and 11 are related to the implementation context (items 7, 9, 10, and 11). Observers were requested to report their observations using a 7-point Likert scale ranging from 1 (extremely negative) to 7 (extremely positive). To explore whether the conceptual distinction of these two components is reflective from the data, principal components analysis (PCA) with varimax rotation was used to summarize the effects of the 11 process evaluation items. Two components were identified with eigenvalues greater than 1.0. In addition, the resulting scree plot of the eigenvalues revealed that the leveling off to a straight horizontal line occurred after the second eigenvalue. These two factors could explain 75.02\% of variance. The components emerged to reflect clearly the factors originally proposed. The items from each subscale were loaded on the intended components. Consistent with the conceptual model, two components were formed, namely, implementation process (items 1, 2, 3, 4, 5, 6, and 8) and implementation context (items 7, 9, 10, and 11). The internal consistency of the overall IPC, as shown by Cronbach’s a, was .93. The inter-rater reliability of the IPC, as shown by Pearson correlation, was .87 (p < .001). The inter- nal consistency of the Implementation Process subscale was .92 and that of the Implementation Context subscale was .82. Process Outcomes Two items were used to evaluate the observation outcome: implementation quality and implementation success. Observers were requested to indicate their observations using a 7-point Likert scale ranging from 1 (poor) to 7 (excellent). A higher score represents better implementation quality or success. The inter-rater reliability for implementation quality, as shown by Pearson correlation, was .73 (p < .001), whereas that for implementation success was .69 (p < .001). Results The inter-rater reliabilities of the scores were high, allowing the ratings of each item by the two observers across all units to be averaged. Table 3 shows the descriptive profile of the evalua- tion indicators for process evaluation. The overall program adherence to the established manual ranged from 12.5\% to 95.0\%, with an average overall adherence of 76.18\%. All other items used the 7-point scale. We set 4.50 as the cutoff point as an indication of high or low rating; it is a more stringent criterion instead of using the mid-point. This can differentiate 542 Research on Social Work Practice 21(5) some factors from others and provide a more balanced picture. The scores for implementation quality and success were 4.63 (SD ¼ .94) and 4.68 (SD ¼ .82), which are high. The scores of the 11 process evaluation items ranged from 4.48 to 5.60. Classroom management (5.60) and familiarity with students (5.40) had the highest scores, whereas reflective learning (4.48) and time management (4.55) had the lowest scores. Apart from reflective learning, all scores were on the high side. The 11 items were divided into the two groups of the PCA: implementation process and context. The mean score for implementation process was 5.03 (SD ¼ .97), whereas that for the implementation context was 4.95 (SD ¼ .99). Both scores were on the high side. Table 4 shows the inter-correlations among program adher- ence, implementation process, implementation context, imple- mentation quality, and implementation success. All variables were highly related to each other. Quality versus success (.98) and process versus quality (.83) had the highest correlations, whereas process versus adherence (.51) and process versus context (.67) had the lowest. In addition to correlational analysis, multiple regression analyses were also performed using program adherence, implementation process, and implementation as independent Table 1. Summary of the Program Objectives of the Observed Units School Program Units Program Objectives A MC 3.1 To discuss the differences between fairness in our ideals and in reality To understand that a system or situation of ‘‘absolute fairness’’ does not exist in reality MC 3.2 To learn how to exercise self-reflection and how to help others To discuss ways of helping others in society B PN 3.1 To understand that prosocial and moral consideration and analysis are essential when making decisions C PN 3.2 To understand that society has different expectations of different roles To investigate the potential conflict between being prosocial and socially accepted behaviors D RE 3.3 To state how Mencius looked at adversity To reflect upon oneself and how Mencius’ teachings can be applied in daily life RE 3.4 To construct a vision of one’s future family To recognize that one needs to work hard and use resources properly so as to achieve their aspirations E BC3.1 and BC 3.2 To understand the importance of forgiving others sincerely To learn how to forgive others for their offenses against us To learn how to observe and appreciate people and things around us F BF 3.1 To adopt a realistic and positive attitude in exploring future careers G BC 3.2 To understand the importance of sincere forgiveness To understand the negative influence of taking revenge on those who have offended us H BC 3.1 To understand that appreciation brings joy to oneself and others. To learn how to observe and appreciate people and things around us, and to express sincere appreciation To learn how to respond to appreciation in a proper manner BC 3.2 To understand the importance of sincere forgiveness To understand the negative influence of taking revenge on those who have offended us I BF 3.2 To understand that different jobs have different requirements To be aware of the issue of gender stereotypes and their impact(s) on career choices J BC 3.1 To understand that appreciation brings joy to oneself and others. To learn how to observe and appreciate people and things around us, and to express sincere appreciation To learn how to respond to appreciation in a proper manner BC 3.2 To understand the importance of sincere forgiveness To understand the negative influence of taking revenge on those who have offended us K SE 3.1 To understand that successful wealth management relies on the ability to exercise self-control and delayed gratification To understand the importance of controlling desires for unnecessary material things SE 3.2 To understand the meaning of dreams and their importance in life To identify the personal qualities that help one overcome environmental constraints and realize dreams L MC 3.3 To learn to cherish love relationships and to love with commitment instead of quitting easily To discuss the proper attitudes to end a love relationship M SC 3.3 To understand the reasons for conflict among siblings To learn the proper attitude to get along with siblings N SC 3.3 To understand the reasons for conflict among siblings To learn the proper attitude to get along with siblings SC 3.4 To understand the reasons for conflict among friends To learn how to face and handle conflict with friends Note. MC ¼ moral competence; PN ¼ prosocial norms; RE ¼ resilience; BC ¼ behavioral competence; BF ¼ beliefs in future; SE ¼ self-efficacy; SC ¼ social competence. Law and Shek 543 variables. Implementation quality and implementation success were used as two separate dependent variables. Table 5 shows the results for the prediction of implementation quality. Both implementation process and program adherence could predict the quality with a large variance explained. Implemen- tation context could not predict the quality. The effect size for the implementation process (b ¼ .51), Cohen f 2, was .35, which is large. The effect size for program adherence (b ¼ .34) was .13, which is medium. Table 6 shows the results of the prediction of implementation success. Implementation process, implementation context, and program adherence could all predict success with a large variance explained. The effect size for implementation process (b ¼ .47) was .28, which is large. The effect size for both context and adherence (b ¼ .29) was .10, which is medium. Discussion and Social Work Implications Project P.A.T.H.S is a huge evidence-based project of positive youth development in Hong Kong. This article attempts to examine the adherence and quality of implementation of the Tier 1 Program (Secondary 3 curriculum) of Project P.A.T.H.S. in the second year of the FIP. We find that the range of program adherence is wide from 12.5\% to 95.0\%. There are various reasons for the difference in the range of program adherence. First, some program mate- rials are overpacked within 30 min. By the time the program implementers have finished part of the program, the lesson time is up. The fact that the implementers cannot complete the pro- gram directly affects the adherence rate. Second, the units observed cover a great variety of constructs. Some schools have already some established programs covering these con- structs before the P.A.T.H.S. Hence, these schools may use these materials instead of manual materials. For example, the construct of ‘‘beliefs about future’’ is related to career planning (i.e., BF 3.1 and BF 3.2). The research team designed some exercises for students to reflect on their career choices. However, schools may have similar career planning exercises from the career and guidance team and may use their materials instead. Another example is the construct of …