Understanding Quantitative and Qualitative Data

Evaluator's Corner

Understanding Quantitative and Qualitative Data

As funders become increasingly "evaluation-savvy," they are requiring more reporting on measurable outputs and outcomes. We live in a number-obsessed society where we are constantly bombarded by statistics, and in order to continue to receive funding for our programs and communicate their effects, we often have to keep up. This need for quantifiable results causes many of us to think of program evaluation only in terms of surveys, statistics, and aggregate data. But this is not always the best approach to data collection for every program. Sometimes, programs lend themselves more to qualitative data collection, which can often provide richer data. Put most simply, quantitative evaluation uses numbers, and qualitative evaluation uses words. People tend to shy away from qualitative evaluation methods for a variety of reasons: they can be more time-intensive, more expensive, and they may seem difficult to transform into results that are easily communicated. The purpose of this article is to explore the differences between quantitative and qualitative data, and to discuss when one or the other may be more appropriate.

Quantitative Data

Quantitative evaluation collects data that can be transformed into numbers in order to conduct statistical analysis. Reports on this type of data quantify the effects of a program on its participants and often discuss the significance of statistical relationships. Although quantitative data may come from different types of data collection (surveys, structured interviews, program or clinic records, etc.), the commonality among quantitative approaches is that they collect information using standardized instruments. No matter how the data is collected, the same information is collected from each subject in a consistent way, and this information is transformed into numerical values. Data are usually collected before and after the program, and results are often summarized using tables or graphs, accompanied by a verbal explanation of the results.

Qualitative Data

Qualitative evaluation uses interviews, focus groups, or a variety of observational techniques to analyze and report on data in narrative form. Other examples of qualitative data collection may involve assessment of written, print, video, or audio documents. For example, one class activity during a pregnancy prevention program might be to have participants keep a journal about their attitudes and values regarding teen intercourse and pregnancy. An analysis of these journal entries might reveal changes in these attitudes and values over time (this process would likely be time-consuming but could produce very rich data). Similarly, one might take video or audiotapes of participants at the beginning of a program (talking about various issues related to the program) and at the end of the program. An analysis of the tapes might suggest how participants changed over the duration of the program.

In qualitative data collection, the evaluator gathers information about the program and how it affects participants, and does not usually rely on statistical comparisons to draw conclusions. Qualitative data analysis often involves identifying recurrent themes and patterns in order to gather insights into a program and its effects.

Advantages and Disadvantages

There are several advantages to quantitative evaluation. For one thing, it allows the evaluator to draw conclusions with a certain degree of confidence. As long as the study has been carefully designed, the evaluator can make very precise statements about the effects of the program, such as the exact percentage of program participants that showed increases in knowledge and favorable attitudes. If the sample has been carefully selected, one can also generalize results to similar populations. A variety of statistical techniques also allow the evaluator to closely examine relationships from different angles to answer complex research questions. Furthermore, quantitative data collection methods are often assumed to yield more objective data than qualitative methods because the evaluator is somewhat removed from the program and participants. Quantitative evaluations are designed beforehand, and do not necessarily develop or change as the process unfolds. But it is important to remember that quantitative studies have their own set of biases. An evaluator decides where to focus, which data to collect, and how to interpret the findings.

While some people may view quantitative data as more authoritative and unbiased, it is important to note that a good qualitative study is carefully planned and methodical as well. A good evaluator gathers information and compares it against other sources, such as information from other stakeholders and program records, before accepting it. When a qualitative evaluator sees a pattern in the data, she gathers additional data to determine whether there is anything that may counter the explanation. In this sense, qualitative evaluations develop in part as they go along. This also allows an evaluator to follow up and investigate further as needed.

One advantage of qualitative evaluation is that the data is often more rich. Qualitative studies can convey detailed information about real individuals, and can illustrate the true meaning of a program. In some instances, qualitative analysis may even be more appropriate. For example, if you have an ongoing program with relatively few participants, it may make more sense to conduct interviews in order gather information about participants' experiences and the efficacy of the program. Unlike a questionnaire, an interview can provide you with data such as testimonials and quotations that can also be used in an evaluation or grant report. Such information can also be invaluable for future funding proposals. The major drawback of qualitative data is that data collection is time-consuming and analysis can seem like an overwhelming task. Furthermore, funders and other stakeholders may be more interested in and convinced by quantitative data.

Which Type of Data to Collect

The choice of whether to use quantitative or qualitative approaches should depend heavily on one's research questions, as well as time and resources. In general, good quantitative data is predictive, while good qualitative data is descriptive. Therefore, when the research questions relate to the processes of a program, qualitative evaluation is often preferable. Qualitative data collection can allow the evaluator to examine the program and its activities and effects while remaining open to new information and ideas about the program. Furthermore, qualitative data can be particularly useful when conducting formative evaluation (evaluation conducted mainly to improve a program). Particularly when implementing a new program, qualitative methods can reveal all kinds of useful information about how a program takes shape and what successes and challenges are met along the way. In general, when one is looking for richness of detail, qualitative methods are very useful. On the other hand, when your research questions relate to specific program outcomes, a quantitative approach may be preferable. Quantitative data can express program outcomes in precise numerical terms. Rather than numbers and statistics, qualitative studies tend to use words such as many and most, and discuss relationships in the data in more general terms.

Overall, quantitative methods are best for answering specific questions about the relationship of program inputs and activities to outcomes; qualitative methods are generally best for understanding the meaning of program processes to different stakeholders, and for discovering unexpected patterns of association among unexpected data. Qualitative data collection may be more appropriate and revealing in some instances, and it is important to remember that using such methods does not prevent you from being able to communicate your evaluation results in ways that are clear and convincing to your audiences. Moreover, a combination of both quantitative and qualitative data can sometimes provide the most complete picture of a program and its outcomes. Where appropriate and when resources permit, do not hesitate to use multiple methods in evaluation.

If you have questions about evaluation, would like assistance, or would like to request an Evaluator's Corner edition on a particular topic, please do not hesitate to contact me: Julia Kohn, Planned Parenthood Federation of America, 434 W. 33rd St., New York, NY 10001. Phone: 212-261-4629. Fax: 212-247-6269. Email: julia.kohn@ppfa.org