Program evaluation is an essential methodical activity in the development of educational strategies, directed at gathering, examining, and interpreting data to determine the success, efficiency, and value of particular programs. These assessments are designed to measure the progress in attaining specific goals, reveal drawbacks, improve program execution, and report whether changes in students’ abilities and knowledge acquisition occurred. In this regard, scholars developed different theories, including reductionism, system, and complexity theory, that contains underlying principles ensuring guidance for the correct evaluation. On the ground of these theories, various evaluation models have emerged, such as Kirkpatrick’s four-level model, Logic model, CIPP model, and quasi-experimental models. Overall, evaluation is a complicated, multifaceted process requiring the consideration of many critical details and a practical approach to gain in-depth insight.
This given paper aims at evaluating a one-week study skills program focussing on developing a range of skills such as reading texts, taking notes, setting objectives, taking tests, and cooperating with educators, among others. The paper will choose a logic model as the most appropriate method for the program evaluation and provide a rationale for its selection. In particular, the approach allows for establishing a coherent connection between the main components of the program’s planning and implementation, including inputs, activities, outputs, and outcomes. The author also composed ten questions related to the study and provide useful information linked to them. Furthermore, the monograph will consider the diversity of the student body via defining the presentation of different races in the project and its impact on the results. Finally, the paper recommends using a PowerPoint presentation as a means of delivering the program’s findings to the institution and the public. The issue of transparency, information disclosure, and privacy has also been raised.
Evaluation Model
A logic model (LM) is applied in the study, which targets defining the connection between the program’s educational techniques and components and the desired outcomes. The given model is built on the change theories describing the row of activities provoking positive shift and their linkage to the expected results. Precisely, the LM consists of the four components, namely, inputs, activities, outputs, and outcomes (“Using a logic model,” n.d.). Inputs imply the material, staff, and intellectual resources necessary for an educational program, while activities include strategies, interventions, and technologies. Outputs assume direct products and services completed or delivered by the programs, whereas outcomes are the short-, medium-, and long-term changes stemming from the program’s activities (Knowlton & Phillips, 2012). According to LaForett and De Marco (2020), the model demonstrated its benefit for addressing racial disparities and interpersonal biases. The other main advantages of the LM comprise its utility during the planning and management phases, ease of use, increased understanding of the program (Linfield & Posavac, 2019). Thus, this method will allow for measuring shifts in the knowledge of learners participating in the study skills program.
Evaluative Questions
What amount of money and time was invested? Were there any deviations from the planned budget?
The answers should elaborate on the planned and final costs of the program and each activity. Besides, respondents should tell about specific changes occurring in expenditures and their causes. The same should relate to the time invested in the course and each activity.
Were program activities realized as projected? Were certain changes to the target activities introduced, and why were they need?
These questions require describing activities’ implementation thoroughly, including obstacles, favorable factors, and detail needing careful and future consideration. In addition, if there were some changes, respondents should specify the main reasons for them.
Did the anticipated staff participate as expected? Who did not and why? How many sessions were missed? Did they possess the required competence and experience?
The replies to these row of questions should contain detailed information about the personnel’s availability, especially teachers, on lessons, the number of lessons missed and why they were not attended. Moreover, interviewees should briefly inform educators’ experience and qualifications.
Did the activities meet the needs of all students, especially those in which the course could be of particular interest? (Giancola, 2020)
Interlocutors can relate the utility and relevance of the projected activities, such as reading texts and course materials, setting goals, working with tutors and electronic tools. Besides, they should indicate whether learners were satisfied with these activities regarding their content and delivery.
What obstacles or problems to the implementation of the program have arisen? How was the program adjusted or corrected to overcome them?
The respondents should tell the evaluator about the barriers that emerged during the program implementation process and decisions made to address them.
How did participants in the course activities estimate the activities for accessibility, effectiveness, benefit, and engagement?
The reporters should give information about participants’ opinions about activities’ accessibility, efficacy, and significance. Respondents also can indicate the methods and tools used to receive participants’ answers, including surveys or online questionnaires.
What were learners’ performance and achievement outcomes? Did students perform the planned curriculum successfully and in full? What activities did they fail to succeed in? (“Specify the key evaluation,” 2016)
The answers to these questions should contain information about students’ attainments in the course and what difficulties and challenges most learners experienced in their acquisition of the program’s material.
How frequently or how efficiently did pupils utilized acquired knowledge and skills in their program practice? Do activities bring a particular practical significance for learners? (Frye et al., 2012)
Concerning these questions, the responses should have information from surveys or questionnaires that inform if students apply the skills they learn during the sessions. Additionally, there should be a clear conclusion on whether activities are practical for students in their educational environments.
How actively was faculty engaged in the sessions? What are faculty members’ impressions and views on the course of the program and its overall benefit? What have they learned from the program’s activities?
These questions require extensive answers, primarily comprising the information about who, besides teachers, were involved in the program (administrators and other professionals) and how active they were. Furthermore, the answers should contain their feedback on the flow of the course, especially its shortcomings and problematic areas needing adjustment and enhancement. Finally, what knowledge have they obtained during the program personally?
How did students’ outcomes concerning their studies change after the program’s attendance?
The final critical question necessitates determining the result the related learners obtained in their studies after a particular period of time, for example, two or three months. The improved learning outcomes on specific subjects will signalize the benefit of the program. On the other hand, insignificant or no change will demonstrate the need for making corrections and additions to the program.
The Interested Parties
The primary parties to which the findings of the assessment would be helpful is the governmental entities and organizations. Serving as funding agencies, they can be seriously interested in the program’s scale-up and its inclusion in a curriculum of other institutions (Stegemann & Jaciw, 2018). The other stakeholders include educational establishments, the administration and faculty of which could want to adopt and develop a similar program in the case of the effectiveness of the evaluated program. Finally, students and pupils could also be much interested in the estimation of such programs to define whether they should attend them.
The Consideration of Inclusiveness
First of all, the assessment would examine whether the program consisted of a diverse student population and the percentage of participants of different races. In addition, the evaluator would collect students’, teachers’, and other professionals’ beliefs and feedback about inclusion (Yarbrough et al., 2010). Finally, the evaluation would assess if the program promoted diversity among the participants and how it impacted the attendees’ attitudes towards other minorities, such as Black, Latino, and Asian.
The Presentation of Findings
The demonstration of the program’s results can be executed using a PowerPoint presentation as the most comfortable, relevant, and illustrative way to reveal all necessary information to the public. Herewith, the given presentation should contain all essential findings in the form of graphs, tables, pictures, and notes that may be the most thought-provoking and valuable for spectators. It is also worth noting that the presentation will show only credible, honest, and complete data, with disclosure of main sources of support, including financial ones (Yarbrough et al., 2010). Finally, presenters should consult with all stakeholders concerning the revelation of their information to avoid possible misunderstandings and conflicts.
Conclusion
The paper has delivered the assessment of a one-week study skills program focussing on developing a range of skills such as reading texts, taking notes, setting objectives, taking tests, and cooperating with teachers, among others. The document has applied the logic model that presents a mapping approach connecting a project’s main components and objectives with necessary resources, needs, and outcomes. Specifically, this model allows for building specific, constructive questions that aim at evaluating all essential facets of the program, including stakeholders’ involvement, the amount of investment, cooperation between participants, and many others. Overall, the given study comprises ten questions that concern money and time, the program activities’ efficacy, students’ performance and achievements, faculty’s engagement, acquired skills, and problems and barriers. The presentation’s result will be primarily interesting for governmental entities and funding organizations, educational establishments and their faculties, and a vast range of students who may pursue improving their cognitive and learning skills.
The monograph also has raised such significant issues as diversity and inclusion that should be an indispensable component of any educational program. In particular, the evaluation should check the availability of a diverse student population in the program, participants’ attitudes towards minorities, and how these views have changed after completing the course. In this regard, the assessment would contain specifically designed questionnaires concerning diversity, which would be delivered before and after the program to compare answers and make respective inferences.
Finally, the study’s results will be displayed in the form of a PowerPoint presentation comprising all necessary charts, pictures, and tables to deliver the data in a comprehensible and engaging manner. Herewith, the notion of transparency and disclosure will also be integrated into the presentation as the fundamental requirement. To avoid various potential conflicts, misunderstandings, and violations of somebody’s privacy and other vital human rights, a thorough consultation will be conducted with all participants on the most sensitive issues.
References
Giancola, S. P. (2020). Program evaluation: Embedding evaluation into program design and development. SAGE Publications
Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE guide no. 67. Medical Teacher, 34(5), e288-e299. Web.
Knowlton, L. W., & Phillips, C. C. (2012). The logic model guidebook: Better strategies for great results. Sage Publications.
LaForett, D. R., & De Marco, A. (2020). A logic model for educator-level intervention research to reduce racial disparities in student suspension and expulsion. Cultural Diversity and Ethnic Minority Psychology, 26(3), 295. Web.
Linfield, K. J., & Posavac, E. J. (2018). Program evaluation: Methods and case studies. Routledge.
Specify the key evaluation questions. (2016). BetterEvaluation. Web.
Stegemann, K. C., & Jaciw, A. P. (2018). Making It logical: Implementation of inclusive education using a logic model framework. Learning Disabilities: A Contemporary Journal, 16(1), 3-18.
Using a logic model. (n.d.). The Pell Institute for the study of opportunity in higher education. Web.
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users. Sage Publications.