Planned Parenthood

Planning Evaluation

Planned Parenthood Educators: Program Evaluation Tools: Planning Evaluation

How can I evaluate my sexuality education programs?

Solid program planning provides a strong foundation for program evaluation activities. Planning is an essential part of program and session evaluation, just as evaluation must be a part of program and session planning. It is impossible to evaluate a program or session for which a plan was never developed. And, it is important to identify evaluation activities to use while you are planning your program or session. Plan to include classroom learning activities that will serve as assessments of student learning. They will provide you with feedback throughout the session about what students are learning and understanding. This approach keeps evaluation activities integrated in your work with students.

Planning Evaluation

  1. Start by clarifying what you hope your program or session will accomplish. Be realistic about your stated goal. Base your plan to achieve your goal on the needs of the students, including your assessment of

    • who they are
    • where the session will be
    • how much time is available
    • what students already know
    • why this program or topic is scheduled now
    • anything that helps you tailor the experience to their unique needs
  2. There are different approaches to program planning that can be effective. Typically, you will begin by writing a goal statement and identifying learning objectives for your sessions or program. Use verbs and write goals as actions. Goals grow from the students’ needs and guide your objectives. Objectives are measurable, specific, reasonable, and pertinent to your goal. Well-written program and session objectives can guide your evaluation activities. To structure objectives that will be useful in evaluation activities, ask yourself

    • Do the objectives correspond to the needs of the students?
    • Are objectives specific enough to be useful as guides for planning and as measures for progress?
    • Are objectives specific enough to communicate clear, unambiguous expectations to participants and others?
    • Are objectives stated in terms of learner outcomes?
    • Are program and session activities appropriate to realize the stated objectives?

    You can also use other program planning models. One branded curriculum development model that is very comprehensive is called Understanding by Design (UbD). Adapting a UbD planning format for sexuality education program planning works well because it focuses on whether the students understand the sexuality education content and how it can be applied in their life, rather than seeking simply to increase what students know about a topic. Ubd is based on the work of McTighe and Wiggins. When using UbD, you begin by identifying the “enduring understanding” (EU) for the session or program. The EU is what you want your students to take away and retain after they participate in your program. Next you list the “essential questions” (EQs) that your students will answer in the session to demonstrate that they understand the topic that is being addressed.

    Writing these goals and objectives or EU and EQs for each session will help you stay focused on what you want your students to gain from the session or program. This is especially important for programs of short duration, which require sharp focus to maximize the limited time available for sessions with students.

  3. Design your session or program plan using learning activities that will appeal to diverse learners, and are developmentally appropriate to your group. Each learning objective or EQ you hope to address in the session should have a corresponding learning activity. Use activities that allow your students to demonstrate what they are learning during the session, so you can adjust your efforts accordingly. Experiential learning techniques can help students quickly identify takeaway messages and application ideas for real life. There are three key questions that help identify learning and application possibilities in any experiential learning activity that students experience in or outside the classroom. The questions are

    • What was the experience?
    • What can we learn from it?
    • What will we do with this knowledge?
    Experiential learning discussion questions only take a few minutes, enhance learning substantially, and provide you with valuable information about student knowledge and skills. Be sure to include experiential learning questions in your lesson plans.

  4. Using pre-test surveys before starting your program measures what students know and think when they enter the program. Review and compare students’ responses on pre-test surveys with your lesson plans to be sure you will address students’ needs and interests. In some cases, you may not have the flexibility to base or adjust programming on your students’ pre-test information. When replicating a science-based program with fidelity, for example, it is necessary to adhere strictly to curriculum protocol. In such cases, the standardized curriculum and program plan must be utilized. It diminishes the opportunity to tailor your curriculum and program to the needs of a particular group of students Typically this less flexible approach is limited to projects involving at least a quasi-science-based evaluation plan.

  5. During the session and throughout the program, record data about
    • attendance
    • variations you make from your lesson or program plan 
    • the extent students can respond correctly to EQs during your session
    • students’ ability to identify and apply the knowledge you have shared with them
  6. Solicit student feedback informally and check in again at the conclusion of the session or program — and perhaps later. Explore
    • how students’ thinking or attitudes may have changed
    • how students benefited from participation
    • what worked for them and what might be improved
  7. Use written surveys at the conclusion of the session or program to determine
    • how much students enjoyed the session
    • what they gained from participating 
    • if they can demonstrate knowledge gains 
    • if they can demonstrate successfully achieving session or program objectives
    • what program changes they might recommend
    • what else they hope to learn
  8. Review and share the information to shape your work with these and other students in the future. Many stakeholders will benefit from sharing data from your evaluation activities. Formal and informal reports can be provided to

    • inform supporters and funders about achieving stated program goals and session objectives
    • inform staff in order to make adjustments in programming to better address students’ changing needs
    • facilitate strategic decision making among administrators about program funding and oversight
    • update and inform participants about their individual and group progress, achievements, and needs

What tools can I use for doing evaluation activities?

There are many tools for doing evaluation activities. They include

  • questionnaires 
  • surveys and pre- and post-tests
  • reports and diaries
  • interviews
  • observations
  • activities and games
  • video and audio taping

To best choose your methods, consider who will be the users of your evaluation data. Many individuals may benefit from knowing more about how your sexuality education programs and sessions are going.

Questionnaires, Surveys, and Pre- and Post-Tests

Things to consider when using written tools

  • Consider using already existing surveys. There is no need to start from scratch when evaluating your programs. Published instruments are available and have added reliability. Crafting clear questions that provide valid and useful data can be difficult. Using what is available from program evaluation experts can increase everyone’s confidence about your evaluation activities.
  • Sometimes, developing or tailoring your own instrument is your best approach. This allows you to assure that you ask only about content that will be included in your program. The unique language and needs of your students can be addressed. Reviewing several published surveys for ideas can be very helpful.
  • Ask only about topics and content you will cover. Knowledge, attitudes, or behavior won’t change about pieces you don’t cover. Unnecessary or “off-topic” questions confuse data and are irrelevant to the “bottom line.” 
  • Accommodate variations in reading and language skills within the group. Reading the survey questions aloud can help to “standardize” the experience for learners.
  • Be sure to address all concerns about confidentiality. Provide ways that students can respond to surveys anonymously, or ways that do not link their identities to the survey. Use identification numbers for each student to allow an individual’s pre- and post-test or annual survey scores to be compared and tracked over time.
  • Obtain informed consent for participation in evaluation activities from those participating in the survey, session, and program. Formal consent is sometimes required from students’ parents or guardians. State laws as well as school and organizational policy usually dictate this.
  • Pre-tests can provide essential data about your students’ more urgent learning needs. Review pre-tests as you are planning a series of sessions for your students; if you find most students more often incorrectly answer questions about one or more topics than others, shift more attention to those topic areas.
  • Scores and performance are affected at both the individual and group level by student attendance. Students who attend more program sessions, or who get a higher program “dosage,” will generally experience more benefits from the program. Low group attendance rates for a program overall, will negatively affect program outcomes.
  • Pre- and post-tests and follow-up surveys can measure changes within the group as well as changes for individuals.
  • Data from these tools provide valuable information for session facilitators, students, program administrators, funders, parents, and other caregivers.

Reports, Diaries, and Interviews

  • You can capture valuable qualitative information about the individual experiences students are having by using written reports. Reports and dairy/journal entries can be recorded in hard copy notebooks or Web-based files. Daily or weekly diary entries about selected topics or identified behaviors can be a rich source of evaluation data.
  • One-on-one or group interviews can provide important information about students’ perspective on a topic or situation. Over time, changes in students’ communication style and in the content of their reports can also indicate growth and progress.
  • It is vital to assure and maintain confidentiality for all written and recorded data. Let students know who will have access to seeing or hearing the information they provide.
  • Recognize that it makes a difference who asks survey and test questions. It can be helpful to use individuals who are trained to interview and who do not have an existing relationship with the students.

Observation, Activities, and Games

We can often observe changes among our students over time. To maximize objectivity in these observations, use a rating scale that clearly describes each category for different performance levels of a desirable task, e.g., communicating about condoms. Rating scales allow you to observe and rate the extent that students can demonstrate skills and increased abilities over time. An opportunity for using this approach could be to observe a role-play among students. If in our example, your objective is to increase students’ ability to provide factual non-biased information to peers, your rating scale might look like the following:

1

2

3

4

Demonstrated nonjudgmental attitude consistently. Offered only factual information.

Demonstrated nonjudgmental attitude with a few exceptions. Offered opinion, but identified it as such.

Demonstrated minimal judgmental reaction. Offered some opinions, but was open to listening to other person.

Demonstrated judgmental and opinionated attitude with regularity.

 

Games and activities can provide an opportunity for students to demonstrate learning as individuals or in groups. Examples include matching the definition of body parts to the anatomical name, or asking students to design an ad campaign encouraging consistent condom use.

Video and Audio Taping

We can use meaningful and objective methods to evaluate and document behavior changes, skill development, and improved performance among our students. Video and audio taping can allow a side-by-side demonstration of performance and communication skills. It can also capture skills your students have developed. Students can view their own performance and gain powerful insight about their effectiveness and skill. It is sometimes necessary to obtain signed consent from parents or guardians to use video and audio taping in your programs.

  • tumblr icon
  • google plus icon
  • twitter icon
  • rss icon
Planning Evaluation