Educational program evaluation is carried out in order to assess the effectiveness of the program itself. This process is crucial in determining the success of a program and its potential in reaching the goals set by its authors. This is done via manipulation of data such as its collection and analysis. Program evaluation mainly assumes five factors: experts, consumers, participants, decisions, and programs involved in the course. This paper is going to use a master’s degree in economics provided by the University of St Andrews to prove the effectiveness of the combined evaluation approach in assessing the given course. The effectiveness of the combined approach to the evaluation of a program can be seen through the application of several models such as CIPP to a given course or degree. The combined approach helps to identify the strengths and weaknesses of a certain program and also aids in establishing a relationship between them.
The economics master’s degree offered by the University of St Andrews is the subject of continuous improvement and evaluation. The university is constantly gathering feedback from the student body as well as third-party members on the quality of the modules involved in the course. Student performance is also a subject of evaluation through numerous assessments carried out throughout the course. This is done through the combination of all the five common assessment methods. The methods and models described below fit the criteria through which the given program can be evaluated.
One of the key models which can be applied whilst carrying out combined evaluation is the Context, Input, Process, Product (CIPP) model. This model is one of the oldest and yet well-tested models available. It was created by Daniel Stufflebeam in the late 1960s with the initial purpose of evaluating school programs in the United States (Anh, 2018). The model is designed not only to evaluate individual aspects but also to establish relations between different factors (Gunung & Darma, 2019). CIPP model helps to evaluate the program in four different ways and, thus, can be effectively used in evaluating graduate programs as it provides an extensive checklist for the evaluator. The model is updated regularly, which only proves its efficiency (Anh, 2018). For instance, the economics master’s degree in St Andrews partially falls under this criteria as it uses surveys to evaluate the context of the material learned as well as continuously analyzing the study material. This can be seen in the quick response to the COVID-19 pandemic and the smooth transfer of studies into online form without any interruptions without damaging the process of the course delivery.
It is also important to realize that the CIPP model has certain flows to it, which make the combined approach to educational program evaluation a subject to debate. Despite being praised for its accuracy and flexibility, it has certain flaws which might need consideration before using it, such as statistical inaccuracies which may arise. It can be argued, however, that no model is flawless and, thus, using CIPP to perform a combined assessment is crucial in improving the learning process (Aziz & Rehman, 2018). Therefore, regardless of its flaws, the CIPP model remains one of the most efficient in examining educational programs using various methods at once.
Another crucial factor in educational program evaluation is its effectiveness for the consumers of the program. One of the key ways to measure it is to analyze the achievements of the students who are undergoing the program. It can be done by looking at the potential career opportunities offered to those who graduated from the course. According to Abdulla and Mohammed Hussain, every institution must go through a three-step evaluation cycle which involves approving the set of learning outcomes, then evaluating the achievements, and finally improving the process (2019). This type of evaluation is mainly conducted via the extraction of alumni data from resources such as LinkedIn. By combining these three steps together, a university can ensure effective course delivery. This will result in improvements in the program as well as a higher level of satisfaction among the consumers.
Constant evaluation of the participants of the process is also required in order to make an effective assessment of a program. Members of staff (especially those delivering the course) should be held accountable for their job (Astramovich, 2016). The evaluation of the teaching process is crucial in establishing the strengths and weaknesses of a certain program and the activities it offers (Darma, 2019). This requires an efficient evaluation method of a program evaluation. This can be carried out through “Value-added models” (VAMs), which were designed to estimate the effectiveness of those teaching the course (Darling-Hammond et al., 2012). The model focuses on individual teaching performance and links student’s achievements to the quality of teaching. However, it disregards the fact that a teacher may be disadvantaged based on the students they teach. This brings the need for the combined assessment approach, which could evaluate both: the effort and willingness of those undertaking a particular model and the quality of the material delivery.
To conclude, the observation shows that the combined assessment approach in evaluating an educational program tends to be more effective than using assessments methods individually. This can be seen through the success of the CIPP model, which has been in use for more than half the century. Many institutions tend to use it or at least use certain aspects of it in order to reach the highest quality of services provided. The economics degree used in the example is not an exception since the university is constantly providing, gathering, and analyzing data regarding it. The way it is done is more efficient than approaching the subjects of the assessment individually. It provides surveys for students regarding the quality of the material taught, such as lecture outlines and the delivery. Furthermore, regular assessments take place among the course material take place in order to check on students’ performance. This proves that combining methods of an assessment provide a wider perspective on the educational program and, thus, give broader results. Including multiple factors in the assessment, such as those offered by the model, will be crucial in identifying a program’s key strengths and weaknesses. The application of individual methods will give results that may be more statistically correct and more limited and may ignore some critical flaws of a program observed.
References
Anh, V. T. K. (2018). Evaluation models in educational program: Strengths and weaknesses. VNU Journal of Foreign Studies, 34(2).
Astramovich, R. (2016). Program evaluation interest and skills of school counselors., Professional School Counseling, 20(1), pp. 54-64.
Aziz, S., Mahmood, M., & Rehman, Z. (2018). Implementation of CIPP model for quality evaluation at school level: A case study. Journal of Education and Educational Development, 5(1), pp. 189-206.
Darma, I. K. (2019). The Effectiveness of Teaching Program of CIPP Evaluation Model: Department of Mechanical Engineering, Politeknik Negeri Bali. International Research Journal of Engineering, IT & Scientific Research, 5(3).
Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. The Phi Delta Kappan, 93(6), pp. 8-15.
Gunung, I. N., & Darma, I. K. (2019). Implementing the context, input, process, product (CIPP) evaluation model to measure the effectiveness of the implementation of teaching at Politeknik Negeri Bali (PNB). International Journal of Environmental and Science Education, 14(1), 33-39.
Hussein, A., & Hussein M. (2019). Social-media based assessment of academic programs., Studies in Educational Evaluation, 62, pp. 149-15