“The public believes, incorrectly, that classroom instruction is as natural as showing your child how to fish or helping a nephew play Ms. Pac-Man. But those comparisons don’t take into account the profoundly specialized discourse of K–12 instruction.”

–Todd Finley

Introduction

After a curriculum is developed, the curriculum committee can breathe a sigh of relief, but their work is not done. Only when the curriculum is implemented and then evaluated will the committee know to what extent their efforts were successful. It is fair to say that no curriculum is perfect because there are almost always factors that may influence the curriculum that were unknown during the development process. As a former curriculum coordinator, I know that every curriculum, if it is to be most effective, is revised based on the results of an evaluation.

Essential Questions

• Why is curriculum evaluation an essential part of the curriculum development process?
• How is monitoring different than evaluation?
• What are the types of curriculum evaluation?
• What are the key elements in writing good curriculum?
• How does a well-written curriculum affect teaching and learning?
• How can the Aligned Curriculum concept help to organize the curriculum development process?

Meaning of Curriculum Evaluation

Curriculum evaluation is the assessment of programs, processes, and curricular products that are resources, not people (Oliva, 2009). There are two parts to the evaluation of the curriculum instruction process. The first is the evaluation of students (most often in meeting the standards) which takes place before, during, and after instruction. The question is, have the objectives been met?  Teachers analyze student assessment data to see how many students have met or not met the objectives, and at what level of performance. The second is the evaluation of the effectiveness of the guides and resources, and the instructor or teacher. This is often done in groups, and over a period of time.

From Curriculum Studies, pp. 87- 93

Evaluation helps to establish the worth of a program and make decisions on whether to continue, stop, or modify the project. The various tools for collecting data and the sources of the data is also discussed.

As with most terms in the curriculum, there are a variety of definitions given to evaluation. Simply described, it is a process of establishing the extent to which the objectives of a program have been achieved by analyzing performance in given areas.  Thus, evaluation is a judgmental process aimed at decision-making. Doll (1992) also defines evaluation as a broad and continuous effort to inquire into the effects of utilizing educational content and process to meet clearly defined goals.

Yet another definition states that “evaluation is a process of collection and provision of data for the sake of facilitating decision making at various stages of curriculum development.” (Shiundu & Omulando, 1992)

Thus, curriculum evaluation refers to the process of collecting data systematically to assess the quality, effectiveness, and worthiness of a program. The process of curriculum development and implementation raises issues like:

• What are the objectives of the program?
• Are these objectives relevant to the needs of the individual and society?
• Can these objectives be achieved?
• What are the methods being used to achieve these objectives?
• Are the methods the best alternatives for achieving these objectives?
• Are there adequate resources for implementing a curriculum?

Certain terms are closely related to evaluation. These include assessment, measurement, and testing.

Assessment, Measurement, and Testing Relating to Evaluation

There is often confusion between the terms: assessment, measurement, and evaluation. The following descriptions help define the differences.

Assessment

Assessment is the process that shows whether there has been a change in student’s performance in a certain academic area. The change revealed through assessment can be given a value by quantifying procedures referred to as educational measurement. Assessments include the full range of procedures used to gain information about student learning and the formation of value judgments concerning learning progress.

Measurement

Measurement is the means of determining the degree of achievement of a particular objective or competency.  For example, the outcomes for individual students are measured (using assessments) to see if the student has met the outcomes.   Measurement refers to the determination of the actual educational outcomes and comparing these with intended outcomes as expressed in the objectives of the program.  Measurement describes something numerically. There is currently a focus on the measurement of 21st century skills for students.

Evaluation

In the above context, evaluation is the process of making a value judgment based on the information gathered through measurement and testing. Evaluation of a curriculum occurs so that the developers can accept, change, or eliminate various parts of a curriculum. The goal of evaluation is to understand whether or not the curriculum is producing the desired results for students and teachers. Evaluation is a qualitative judgement.

Monitoring versus Evaluation

From Curriculum Studies, p. 89

Monitoring

What is monitoring? It is a continuous review of the progress of planned activities. Put differently, it is the routine daily, weekly, or monthly assessment of ongoing activities and progress. Monitoring focuses on what is being done. It is centered on two questions:

• Is the curriculum project reaching the specified target population?
• Are the various practices and intervention efforts undertaken as specified in the curriculum project design?

Monitoring is important in examining the inputs and outputs. Indeed, it can be considered as a “process evaluation.” Monitoring thus helps to ensure that the implementation is on course.

Evaluation

Evaluation in relation to monitoring is the episodic assessment of the overall achievement. It examines what has been achieved, or what impact has been made. Evaluation also examines the gaps in the curriculum in addition to what may have been achieved by the students but wasn’t.

Purposes of the Evaluation

From Curriculum Studies, p. 90

Evaluation can serve as a diagnostic tool for remedial teaching to improve pupils’ learning, but it also serves different aspects and participants in the education process. These include feedback to students themselves, classroom purpose including appropriateness of methods, curricular materials, and even the community itself.

It is also the process of determining whether or not the objectives have been attained; often at the end of the program (summative) but sometimes periodically, during implementation (formative). Monitoring focuses on whether the targeted population is being reached, assessment of the flow of inputs and outputs, plus their adequacy and relevance. Evaluation checks on the attainment of objectives and provides objective data on various aspects of the curriculum and its effect on targeted beneficiaries. Continuous information from monitoring helps to identify weaknesses and strengths in the process (e.g. inadequate resources). This could help in modifying objectives, among other changes.

Evaluation data can improve curriculum development through decision-making, whether to modify curriculum content, methods, proposed teaching, and learning materials, and even evaluation approaches recommended. Timely decisions ensure the right direction is taken promptly in the development process.

Evaluation serves the following purposes:

• Individual student progress purposes:
  • Discover what the students have learned (knowledge, skills, attitudes, and adjustment, etc.).
  • Ascertain the student’s status in class.
  • Discover where the child needs help, and the nature of the help needed.
  • Analyze data to determine what is needed to guide each student’s overall growth and development.
• Classroom purposes – Evaluation provides data that enables the teacher to determine the effectiveness of teaching. It helps in answering questions such as:
  • Which of the objectives has been achieved?
  • Are the methods and activities relevant and practicable?
  • Is re-teaching necessary?
• Curriculum materials purposes – Are they relevant, usable, appropriate, and affordable?
• School-wide purposes:
  • Assess the overall effectiveness of the institution.
  • Reveal over or under-emphasis in individual classrooms.
  • Reveal learning areas needing more attention throughout the school.
  • Assist the school administration and staff in planning for institutional improvement.
  • Provide data useful for school-wide instruction and interventions.
• Community – What are the attitudes and inputs of the community to the curriculum and the curriculum development process?

Essentially, evaluation ensures that strengths and weaknesses are detected at an early stage, thus saving time, resources, and frustration to increase the chances of success of a program.

Tyler proposes evaluation at the end of the program to provide data for making decisions about the curriculum, such as reviewing and modifying, etc. John Goodland, on the other hand, proposes continuous evaluation throughout the entire process of curriculum making. This information provides feedback as the process continues and revisions are expected to be made accordingly and promptly.

Types of Evaluation

From Curriculum Studies, p. 91

There are various types of evaluation including pre-assessment, formative assessment, summative evaluation, and impact evaluation.

Pre-assessment

This process helps to determine whether the students possess the prerequisite knowledge and skills to enable them to proceed with new material. It is useful to have this information at the beginning of a new course, or the beginning of a new year in school. It is also useful for teachers new to a class who have not taught before.

Formative Assessment

Formative assessment is that which takes place during the implementation of a curriculum project or program. It guides and promotes the development of the program by providing data for its improvement.

Note: Formative evaluation needs to take place at all stages of curriculum development and implementation.

Summative Evaluation

This is an evaluation carried out at the end of a program. It facilitates major decisions about whether to continue with the program as it is; expand it, modify it, or stop it all together depending on the extent of success or failure of the program.

Impact Evaluation

Impact evaluation is an assessment of how the intervention being evaluated affects outcomes. The effects can be intended or unintended. The analysis of the impact requires an examination of what the outcomes would have been without the interventions. These evaluations are an aspect of summative evaluation and are usually conducted at the state level because it requires rigorous statistical analysis. They establish the impact of the program on the beneficiaries or recipients of the program, and the community, and are generally reported at the state and/or national level.

Criteria for Curriculum Evaluation

From Curriculum Studies, p. 91

The criteria for evaluating the curriculum generally includes alignment with the standards, consistency with objectives, and comprehensiveness of the curriculum. Relevance and continuity are also factors. Many assessments do not cover the entire range of objectives due to difficulty in assessing some of the objectives effectively and objectively (e.g., the affective domain where value traits such as integrity and honesty are tested through written exams). The psychomotor domain, which helps our brain coordinate physical task such as catching a ball, have objectives that are often inadequately tested due to difficulties in logistics. Even with the cognitive domain, the knowledge involving the development of intellectual skills, only a small portion is usually tested. However, a lot of effort is made to try and ensure quality examinations at least at the summative evaluation level through a vigorous process of developing exams, which go through several stages including group analysis, etc.

Consistency with objectives, i.e., curriculum evaluation, should assess and measure the attainment of the curriculum objectives. The various levels of learning objectives need to be kept in mind as per Bloom’s taxonomy, i.e., knowledge; comprehension; application; analysis; synthesis; and evaluation).

The various domains need to be kept in mind (i.e. cognitive, affective, and psychomotor). For example, from a civic education curriculum, one affective domain objective could be, “Learners display appropriate attitudes towards national patriotism.”

The process of evaluation looks for evidence that such attitudes have been developed (e.g. education concepts for good citizenship are often evaluated in terms of knowledge of the government structure or knowledge of civics). This is inadequate (end of citation from Curriculum Studies).

There are a variety of proposals in curriculum literature on what constitutes criteria for evaluation. One example is Wilson’s Curriculum Preliminary Rating Scale which she used in her graduate class, and could be very helpful when evaluating district or state curriculums. If you choose to use this scale, Dr. Wilson would like for you to contact her via the website and give her feedback.  Curriculum evaluation can be judged by consistency of evaluation with objectives of a project; comprehensiveness, validity and reliability, and continuity.

Also, practical skills, or psychomotor skills in home economics, agriculture, biology, etc. are often evaluated through checking for knowledge of facts on the topics, and not the actual practice (demonstrating, applying, or performing the actual skills). This again is inadequate.

Comprehensiveness

From Curriculum Studies, pp. 92-93

All the objectives of the curriculum are evaluated. Often only the cognitive domain is tested through the recall of facts. To test for comprehensiveness, one could carry out an evaluation of the broad administrative and general aspects of the education systems to find out how good the education system is and how relevant the program is.

• Evaluation pertaining to course improvement is determined through assessment of instructional methods and instructional materials to establish those that are satisfactory and those which are not.
• Evaluation related to individual learners will identify their needs and help to devise a better plan for the learning process.
• Feedback to the teachers can shed light on how well they are performing.

Validity, Reliability, and Continuity

Validity

This criterion answers the question, “Do the evaluation instruments used (e.g. examinations and tests) measure the function they are intended to measure?”

Reliability

Reliability provides a measure of consistency with respect to time (i.e. reliable instruments give the same results when administered at different times).

Continuity

Evaluation is a continuous process; an integral part of the curriculum development process and classroom instruction. Hence, to provide continuous feedback on weaknesses and strengths for remedial action to be taken.

Another set of widely shared evaluation criteria that are applicable in any field are relevance, efficiency, effectiveness, impact, and sustainability.

Relevance

Relevance indicates the value of the intervention or program with others. Stakeholder needs, state and national priorities, international partners’ policies, including development goals.

Efficiency

Efficiency answers the question, “Does the program use the resources in the most effective way to achieve its goals?”

Effectiveness

Effectiveness pertains to the question, “Is the activity achieving satisfactory results in relation to stated objectives?”

Impact

Impact focuses on the results of the intervention (intended and unintended; positive and negative) including social, economic, and environmental effects on individuals, institutions, and communities.

Sustainability

Education provides the way each generation passes on its culture, discoveries, successes and failures to the next generations. If there is not adequate inter-generational education, knowledge and accomplishments cannot be sustained. Education is the foundation for formulating, challenging and disseminating ideas, knowledge, skills and values within communities, nations and also globally.

The term sustainability is often only applied to environmental and community issues. Yet to redefine education, the term can also be applied to creating an atmosphere in the classroom that promotes independent skills and attitudes that can sustain the need for life-long learning. It is a given that teachers are under pressure to cover mandated curriculum standards so that students meet learning expectations. However, students are concerned about having relevant experiences that bridge the classroom and the real world. One way to promote these connections is to create a sustainable classroom community through the teaching of self-regulated learning (SRL) skills. (Gerretson, Ilisko and Fortino, 2010).

The Role of Teachers in Curriculum Evaluation

As pointed out earlier, curriculum evaluation refers to the process of collecting data systematically to assess the quality, effectiveness, and worthiness of a program. For evaluation to be carried out effectively, the teacher has to be involved in the process.

Teachers should provide data on the progress of students and materials. Teachers are best placed to judge the quality of materials, the depth to which the topics have been or will be dealt with, and the sequencing of the topics. They document their experiences and those of the learners. Teachers also need to have skills for observing and documenting their observations, constructing appropriate tests and examinations, and systematically reporting their findings.

Teachers’ involvement with curriculum evaluation is of great importance because they are constantly interacting with the learners, so they know them well, and they have continuous opportunities to collect evaluation data through a variety of means including observation, tests, and measurements. They can also easily assess the relevance, quality, and adequacy of teaching and learning resources.

Educational administrators also have a key role to assist the teachers and to coordinate the teachers’ contributions in the area of evaluation. The subject panels or focus groups that are organized at local levels in some countries can, if properly utilized, be a very effective system of initiating and sustaining teacher participation in this process. Through focus groups, many more teachers can be incorporated in curriculum development and evaluation than is possible through the national panels operated at curriculum development institutions in the various countries. A few questions can be discussed on the role of teachers in curriculum development and evaluation. These are:

• How effectively are the teachers involved in curriculum evaluation?
• To what extent are teachers providing feedback to the district curriculum coordinators?
• Are curriculum coordinators seeking information from the teachers?
• To what extent is the feedback from the teachers incorporated in the curriculum and curricular materials?
• Do teachers have adequate skills, time, and resources for observing, testing, measuring, gathering other relevant data, and documentation?
• How well are teachers prepared for developing curriculum and implementing new curricula?
• How coordinated is the flow of information between teachers, administrators, district coordinators, universities, and state agencies?

The involvement of teachers in curriculum development and evaluation could, in the long-run lead to relevant, cost-effective, and self-sustaining education programs.

In addition to the evaluation criteria mentioned, it is important to keep in mind that what is most important in the curriculum development process is the end product. Leslie Owen Wilson states, “Good curriculum development and instructional design should always start with key questions. Where do we begin when we start the curriculum development process?”

Owen Wilson has good advice for those who develop curriculum and urges us to guard against the “for-profit” entities, as well as some of the pitfalls of commercial curriculums that she describes in The Instructional Design/Curriculum Development Process.

Finley (2013) adds that the public has no idea how much mental muscle curriculum planning requires, but corporations that sell education materials do know. Unfortunately, their support is often in the form of selling a “sure-fire” scripted curriculum that “guarantees” growth (if teachers will just follow directions). There are, however, several reasons why scripted curricula-in-a-box do not work:

• What works in one classroom often will not work the next period, so flexibility, intuition, and judgment calls by instructors are needed.
• Values and motivations vary by classroom.
• Prepackaged curricula undermine teachers’ professionalism and agency.
• Cultural sensitivity does not come in a package.
• All students are not at the same level of development.
• Scripted lessons interfere with the all-important teacher-student relationship.

Unfortunately, prepackaged programs are often used with students from low socio-economic populations. For all these reasons, it is best when teachers develop their own lesson plans.

As an alternative to prepackaged programs, teachers can become the center of a curriculum evaluation or review process if they are supported using a collaborative model. Instructional technologists at an international school described in the following article were able to incorporate 21st-century skills instruction that was relevant to the schools. The process led to some innovations in classrooms as well as discussions that focused on a few standards and essential questions that allowed assessments to be uniform as well as options for differentiation of curriculum that were supported by technology.

Summary

Evaluation of the curriculum is a critical phase in the curriculum-development process. Even if all the steps are followed in the development process, it is when the curriculum is implemented that it becomes clear whether or not the objectives have been met and to what degree the students have made progress academically. This is a meaningful, but complicated process. Publishers know this, and in response, they have developed pre-packaged curricula that have many drawbacks. Guarding against these “for-profit” entities is essential in developing a good curriculum that is designed for the success of the students. As an alternative to pre-packaged programs, teachers can be involved in the evaluation or review process if they are part of a collaborative process.