Evaluation: Getting the Help You Need

Evaluator's Corner

Evaluation: Getting the Help You Need
by Julia Kohn, Evaluation Manager

As many of us know, program evaluation can sometimes seem like a daunting task. Often human service professionals recognize the importance of evaluating one's programs, but lack the skills and expertise to plan and conduct an evaluation. Whenever possible, it is a good idea to involve a trained evaluator in your program, preferably during the planning stages. Whether you are starting a new program, or whether you have been running a program for years, an evaluator can help to define program goals and objectives and develop systems to measure a program's success.

Finding an Evaluator

It can be difficult to know where to begin when looking for an evaluator. There are four main sources for professional evaluators:

• colleges and universities;
• program evaluation consultants;
• in-house evaluators who are employed by your organization; and
• firms that specialize in evaluation.

No matter where you begin your search for an evaluator, be sure to look for someone who is trained in statistics and research methods. In colleges or universities, try contacting the psychology or sociology departments or the school of public health or administration. Master's or doctoral candidates may be interested in evaluating your program as the focus of a thesis or dissertation. University faculty may also work as consultants and may evaluate your program for a fee.

As mentioned above, there are also individual consultants and research firms who specialize in evaluation. The former are typically individuals working alone while the latter are part of a larger firm with other staff and administrative support. Some organizations hire full-time employees to evaluate their programs. While some people may argue that independent evaluations are more credible than those conducted by employees, others maintain that there is no real reason to suspect the integrity of an in-house evaluator.

One of the most important qualifications to consider when hiring an evaluator is prior experience with program evaluation. While research methods are similar across disciplines, program evaluation differs from academic research to a certain extent. The following are some tips for hiring an evaluator:

• look for experience in program evaluation, preferably in your area of programming (e.g. increasing sexual health knowledge)
• ask to see a recent report in which she or he played a major role to make sure it is readable and thorough (and understandable to non-technical audiences)
• find out how much time the evaluator has available for your project and who will actually be doing the work
• ask about costs, including overhead and travel (a local evaluator will incur less costs associated with travel)
• discuss and agree upon ownership of the data and publication rights and conditions (if the evaluator intends to publish the results, be sure to negotiate your role)

The issue of cost is probably the most frequently cited barrier to program evaluation. Therefore, it is crucial to build evaluation into a program budget. You will never have the money for evaluation unless you think strategically and build evaluation into your work as an essential part of any program budget. Furthermore, the cheapest evaluator may not always be the best option, and it is very important to consider expertise and the factors described above in addition to cost. Evaluation does not have to be intimidating and overwhelming. Finding an evaluator you like and trust often makes for a better overall evaluation experience. You should be able to understand all aspects of a program evaluation design whether or not you have a background in research. Finally, you should never accept an evaluation design that you cannot understand or that does not feel right to you. Evaluation can be an exciting and illuminating experience, and finding the right evaluator is critical to carrying out a good evaluation.

Other Resources:

Sociometrics publishes a local evaluator consultant's directory that can be purchased for $6. Visit http://www.socio.com/ or call 1-800-846-3475 for more information.

If you have questions about evaluation, would like assistance, or would like to request an Evaluator's Corner edition on a particular topic, please contact Julia Kohn, Planned Parenthood Federation of America, 434 W. 33rd St., New York, NY 10001. Phone: 212-261-4629. Fax: 212-247-6269. Email: julia.kohn@ppfa.org