Assessing Attitudes

Evaluator's Corner

Assessing Attitudes

by Julia Kohn, Evaluation Manager

What are attitudes?

Change in attitude is a commonly measured outcome for sexuality education programs. An attitude is an evaluation or judgment of an object (i.e., a person, place, thing, behavior, etc.). The term attitude is also often used to refer to values, opinions, feelings, and beliefs. Many programs aim to change participant attitudes. Some examples include changing participant attitudes toward safer sex or contraception, and attitudes toward school or school achievement. When it comes to evaluating such programs, measuring attitudes is not simple. In fact, attempting to demonstrate attitude change, is one of the most difficult evaluation tasks. This is the case for several reasons.

Why are attitudes difficult to measure?

First and foremost, the concept of attitude is a construct, created so that we can find order in what people say, do, and think, and therefore can make predictions about their future behavior. We cannot measure or examine an attitude in the same way we measure other things. Instead, we can only infer that a person has a particular attitude by his or her words and actions. For example, Mario says that he hates using condoms. He rarely ever uses condoms during sexual intercourse and says that they take away from the experience, ruin the moment, and are annoying to purchase. From this we can infer that Mario has a negative attitude toward condom use. Like Mario, John also says he doesn't like using condoms. He too thinks it ruins the mood and creates an awkward moment. However, he uses condoms consistently and has never had unprotected sex. It may be somewhat more difficult to describe John's attitude toward condom use.

Attitudes can be complex, with many different facets and manifestations. Some theories propose that attitudes have three components: 1) cognitive; 2) affective or emotional; and 3) behavioral. Sometimes behaviors, beliefs, and feelings may not match, even when we correctly assume that they reflect a single attitude. Therefore focusing on only one manifestation of an attitude may distort your understanding of the situation. Furthermore, a one-time measurement may not be adequate for a volatile or fluctuating attitude. Assessing an individual or group more than once when possible can help to detect inconsistency. All of these factors can make it difficult to accurately measure attitudes in an evaluation, but this does not mean we should back away from the challenge, because we also know that attitudes can play a very influential role in sexual behavior and practices.

Methodology

If you decide to use a self-report measure of attitudes (where the participant indicates what their attitude is) you must be sure that they will be aware of their attitude and that they are likely to be honest about their attitude. Furthermore, when using any self-report method of assessing attitudes (e.g., questionnaires or interviews), care must be taken to ensure that wording of the items/questions does not unduly influence the participant's response. Questions should be constructed so that it is not overly obvious what the desired or socially acceptable response is. When assessing attitudes as a program outcome, we often hope that participants' attitudes will move toward what we deem the most desirable attitude. When we say that attitudes have "improved," we really mean they have moved closer to our idea of appropriate attitudes. This should be recognized and acknowledged.

When possible, it is usually a good idea to use an already existing tool to measure attitudes. Not only does this save you time, but it also gives you the benefit of other people's experience. Using an existing measure means that it has already been tested and you should be able to obtain some data on its reliability and validity. When choosing or constructing a tool to assess attitudes, it is important to measure both the direction of an individual's attitude (e.g., positive or negative) and the intensity of the attitude (e.g., how positive or negative). For this reason, true/false items are not ideal. Finally, any question about attitudes should have a response that allows the participant to indicate that they do not have an attitude, or that they do not have sufficient knowledge about the object to have formed an attitude. This prevents people from choosing an answer simply because they are forced to, and helps you obtain more accurate results.

Questionnaire items assessing attitudes might take several forms:

Agreement scales

Example: Having unprotected sex with a partner you do not know well is an irresponsible behavior (circle one)

   -2               -1               0               1               2
Strongly       Disagree    Undecided      Agree        Agree
Disagree                                                        Strongly

Semantic differentials

Example: Having unprotected sex with a partner you do not know well is (check one):

   Irresponsible   __   __   __   __   __   Responsible 
                       -2   -1    0    1     2  

Ordered scales

Example: Please check the one statement that is closest to what you believe:

___ Having unprotected sex with a partner you do not know well is harmless
___ Having unprotected sex with a partner you do not know well has risks, but it's worth it
___ Having unprotected sex with a partner you do not know well is tempting, but not worth the risks
___ Having unprotected sex with a partner you do not know well is too risky to even consider
___ I do not have an opinion about this issue

Forced-choice items

Example: Please check the one statement that is closest to what you believe:

___ Having unprotected sex with a partner you do not know well is OK
___ Having unprotected sex with a partner you do not know well is not OK
___ I do not have an opinion about whether or not having unprotected sex with a partner you do not know well is OK

Open-ended questions

Example: What do you think about having unprotected sex with a partner you do not know well?
_____________________________________

Keep in mind that questionnaires are not the only evaluation method for assessing attitudes. Focus groups can also be a very useful method of assessment, particularly because one's attitudes are often formed and changed in interactions with others. As stated earlier, measuring attitudes can be a difficult task; however, a carefully designed evaluation can provide a wealth of useful information about participant attitudes. Attitudes can be an influential factor in determining sexual behavior and can make the difference between engaging in risk behavior or not. Therefore it is important that we think carefully about the attitudes of our program participants and measure them appropriately if we expect to demonstrate change.

If you have questions about evaluation, would like assistance, or would like to request an Evaluator's Corner edition on a particular topic, please do not hesitate to contact: Julia Kohn, Planned Parenthood Federation of America, 810 Seventh Ave., New York, NY 10019. Email: julia.kohn@ppfa.org