Writing an Evaluation Plan Into Your Funding Proposal

Evaluator's Corner

 
Writing an Evaluation Plan Into Your Funding Proposal
by Suzanne M. Johnson Vickberg, Ph.D., former director of programs and evaluation

Conducting a thorough evaluation of your program will require resources. That means you should write an evaluation plan into any proposal (and budget) that you submit to a potential donor. This is a very brief example of what the evaluation section might look like in a grant proposal.

Evaluating the Program

Planned Parenthood is dedicated to delivering the highest quality programs possible. In order to insure that our programs are the best they can be, ongoing evaluation is crucial. We plan to evaluate the proposed program, using both process and outcome evaluation. The process evaluation will assess program implementation and will focus specifically on:

• the number of program sessions offered and the number of adolescents served by the program;
• the level of participant satisfaction and consistency of attendance; and
• the extent to which instructors deliver the program as intended in the written curriculum

The outcome evaluation will assess benefits to program participants, answering four research questions that are based on the outcome objectives described earlier:

• Does the program lead to a significant increase in knowledge about safer sex, and is this knowledge maintained for at least six months after the program is completed?
• Over the course of the program, do participants display a change in their attitudes about the use of condoms and other birth control — do they have fewer negative attitudes about these practices?
• By the end of the program, do participants demonstrate improved negotiation skills?
• Does the program lead to a significant reduction in the number of participants engaging in "risky" sexual behaviors, such as unprotected intercourse, and are these reduced rates maintained for at least six months after the program is completed?

The process and outcome evaluations will each have several components. The process evaluation will involve tracking of program statistics by program staff, focus groups with participants, and observation of, and self-reporting by, instructors. The outcome evaluation will involve written questionnaires to be completed by participants and a comparison group, direct observation of participant role plays, and analysis of participant journal entries. Meetings will be held to inform parents about the evaluation process and descriptions of the evaluation will be sent to each participant's home so that parents may decline their children's participation in the evaluation if they wish to do so.

Process Evaluation

The first process objective (program statistics) will be assessed by having program staff track the number of sessions offered and the number of adolescents served. Staff will also track participant attendance (second process objective). Focus groups will be conducted with participants to determine how satisfied they are with the program, their sense of how the program has influenced them, and their suggestions for improvements (also second process objective). The groups will be held half way through the program (10 participants), and one month after program completion (10 participants). The third process objective (program delivery) will be assessed via written reports in which instructors specify those components of the curriculum that were covered as intended, and those that were not. In addition, observation of instructors by program staff will be conducted at least twice during the program.

Outcome Evaluation

The first and fourth outcome objectives (knowledge and behavior) will be assessed via anonymous written questionnaires to be completed by participants and a comparison group. Comparison group participants will be randomly selected from classrooms that are not receiving the program. These individuals will receive the program after the evaluation has been completed. Questions addressing knowledge and sexual behaviors will be drawn from the PMEDS, an assessment tool constructed by Sociometrics. Participants will complete the questionnaire before beginning the program, immediately after completing the program, and six months after completing the program. The comparison group will not receive the intervention at the same time as the program group, but will complete the evaluation assessments on the same schedule. The second outcome objective (attitudes) will be assessed by ongoing analysis of journal entries written by participants. Journal entries will be examined for evidence of changed attitudes over the course of the program. The third outcome objective (skills) will be assessed using direct observation of participant role plays. Role plays will be conducted during the third program session and again during the last session. Instructors will use a checklist to indicate mastery of skills. Questionnaires will be completed by participants and comparisons will be made in separate data collection sessions. Each individual will be paid $5 for completing the pre-test, $10 for completing the immediate post-test, and $15 for completing the six-month follow-up. In addition, transportation to the sessions will be paid for and refreshments provided.

Interpretation and Use of the Evaluation Results

Both quantitative and qualitative techniques will be used to interpret the results of the evaluation. Descriptive and nonparametric statistics will be used to determine whether knowledge, skills, and behaviors change over the course of the program. Content analysis or "theme identification" will be used to interpret focus group and journal entry data. Evaluation results will be used to improve the program and to publicize its efficacy in the community.