The Nature of Program Evaluation
What is Program Evaluation?
Program evaluation refers to the task of making judgments about the worth or value of a program. It primarily involves determining the extent to which previously established program objectives are being, or have been, attained.
It should be recognized that program evaluation is a type of survey research. As such, the overall experience and validity of an evaluation effort will depend a great deal on the extent to which sound research techniques are applied.
Reasons for Program Evaluation
The fundamental reason for evaluating vocational education programs is to seek program improvement. Evidence about the relative merits of a program enables educators to make more rational, data-based decisions about their programs.
Students are the ultimate beneficiaries and consumers of the educational process. Educators have an obligation to ensure that the educational process will deliver what is specified in the program objective.
In addition, program evaluation is a necessary function since the public fiscal budget for vocational education continues to rise. Educators must assure the community that the tax dollar is spent efficiently and effectively. Increased emphasis on program accountability when state and federal monies are expended has also increased the need for well-conducted local evaluations to obtain evidence of accomplishment, and justification for the program dollars spent.
For many years, the educational community has tended to manage its educational programs on the basis of hunches, authority, tradition, and personal experience. The public increasingly demands valid and reliable information concerning the educators' theory that the programs with certain characteristics will yield certain outcomes. The public wants proof that educational programs are adequately and efficiently serving the needs of the community.
Who Should Evaluate?
Programs of vocational education are being evaluated every day by the people they are serving. Evaluations are made by parents, educators, taxpayers, and other people affected by the vocational education program. These evaluations may be accurate or inaccurate, depending upon the amount and source of information used. An important guiding principle of locally conducted evaluations is that they should be made by those affected by, and those involved with, the education program. This includes teachers, administrators, supervisors, counselors, students, parents, alumni, employers, and employees.
Many evaluations are conducted by third-party. This third party evaluation is often conducted by a team of experts who visit the educational agency, observe the program, talk with students and teachers, and report their findings to the professional personnel and the governing board.
Characteristically, third-party evaluation teams have assessed the ways and means an institution employees in attaining the proper objectives, (e.g., instructional process and procedures) rather than emphasizing the significance of program outcomes. For example, the concern has often been with the amount of classroom or laboratory space available, the quality of the physical facilities, the qualifications of the teacher, etc., rather than whether program graduates are employed and able to advance in their career fields. Third party teams can provide another perspective of the vocational program that is useful, however. Hence, these evaluations, even though generally process-oriented, do provide another valuable source of feedback about the vocational program.
In conducting locally directed program evaluations, it is important to have the involvement of those key individuals and groups directly concerned with the program (e.g., teachers, administrator, advisory committee members, school board members). Their involvement from the beginning in appropriate ways generally increases their understanding of the process and their commitment to carry out as many recommended program improvements as possible. Failure to involve them, on the other hand, can result in a fine evaluation study being ignored because it lacks the support of those key individuals and groups who can bring about change and Improvement.
Involving Key Groups
In conducting a program evaluation, it is important to have the involvement of certain groups. The first group to involve is the school administration. This should include the school superintendent, building principals, coordinators, and supervisors. Gaining the support of the school administrators is the first step in securing support and commitment from others, and in ensuring continuing enthusiasm for program evaluation. Without strong administrative support, the evaluation effort is not likely to be successful.
A second key group to involve is what might be called a staff steering committee. The staff steering committee should be composed of all of the teachers and staff directly involved with the program to be evaluated and a person from the school administrative staff. It is also wise to include members of the guidance staff and representatives of teachers from the general education curricular areas.
The steering committee can provide direction and assistance for the evaluation effort in several ways. Members may help devise the overall evaluation plan, develop or update a philosophy statement for the vocational program, review program objectives, develop follow-up and other instruments that may needed, help and conduct parent and/or employer interviews, etc. Often, the staff steering committee is divided into subcommittees with specific responsibilities for planning and conducting the follow-up study or other evaluation tasks.
Other key groups to include in an evaluation effort are the vocational advisory committee(s) and the student committee(s). The involvement of these committees is necessary if you are to gain the support of the community and the students your program serves. It is especially obvious that since students are both involved and affected by the vocational program, they should be highly qualified participants in the evaluation process.
Students of a single or several vocational area, depending on the scope of the evaluation effort, may be involved in several appropriate ways. They can assist with the development of follow-up questionaires or letters, suggest changes in program goals and objectives based on their own experience, etc. Students may serve on evaluation committee which are established or as members of a separate student committee.