What is a Rating Scale and are there any guidelines?
updated 4 months ago
Rating Scales are commonly used survey question types. They are useful because they allow comparison of responses from one question to another. Rating Scales contain "Values" that allow report calculations and "Labels" to define the scale item for interpretation and make answering easier for respondents. Rating Scales make it easier to compare responses across survey questions (or statements).
When creating rating scales, the goals should be to make the scale:
- Easy to interpret
- Interpreted similarly by every respondent
- Able to differentiate respondent views as validly as possible
- Return reliable data (meaning, if it was asked again, respondents would give you the same response)
- Has points that map its underlying idea as closely as possible.
How to add a Rating Scale
The Rating Scale list is the place where regularly used survey scales can be created and stored ready for use in surveys.
To create a new Rating Scale, click on the "+ New Rating Scale" button at the top of the page.
How to write good survey scale questions
Survey scale questions are useful because not only do they tell you what people think, but they also show you answers in relation to each other.
This means that you can compare single answers, or groups of answers. The options for grouping answers means that you can see how groups of respondents (say, by demographics like age, gender, or location) think about what you’ve asked.
Scale questions do all of this in one simple question. This means that when you use them, you are potentially reducing the length of your survey.
It also means you really want to get them right.
Survey scale questions: Overview
The best survey scale question give a full set of options, within the most relevant length. It will make the data you get more accurate, and keep completion times low.
Here is a quick-reference guide for the length and outcomes of survey scales:
- 4-point scales: Result in inaccurate data
- 5-point scales: Result in accurate data, and meets needs most of the time
- 7-point scales: Highly accurate results, but most useful for data that is weighted either positive or negative
- 9-point scales: Result in accurate data, but be wary of them because they can tire people out.
- 10-point scales: Tiring questions without a neutral option; avoid them whenever possible.
How many scale points should my survey have?
The short answer to this question is, ‘it depends’. With each scale, the aim is to give respondents a way to differentiate themselves without cluttering the scale.
Five point scales are most popular followed by seven point scales. These scale ranges tend to work best. Beyond a 7 point scale participants have trouble responding the same way if they repeated the survey, so we recommend staying away from anything above 7 points.
Here is an example of a 5 point scale:
- Strongly disagree
- Strongly Agree
You may also consider providing a “Don’t know” option or “Prefer not to answer” option with a value of say “0” which is excluded from any response calculations.
Odd-numbered scales are the best to use, because they have an even number of options either side of the middle. If you were to create a four-option scale, it would look odd and not seem quite right.
The other benefit of an odd-numbered scale is that it allows you to add a neutral option. If you remember our article about writing good survey questions, you’ll know that giving people ‘wriggle room’ is a good idea.
Use a 5-point scale if you’re pretty sure that responses will be fairly evenly positive or negative.
Use a 7-point scale if you suspect responses will cluster towards one end or the other.
Consider including a “Don’t Know” option
In addition to the 5 points or 7 points on these scales, a “Don’t Know” or “Prefer not to answer” option can be important. Rather than forcing respondents to provide an answer, this additional scale still provides an odd number scale and a neutral score along with an option for people who may not have enough information about the survey area to say so.
Even-numbered questions force answers to be positive or negative
You might find it surprising to learn that scales that force a negative or a positive answer can be frustrating to respondents. It’s because they can make them feel like they aren’t able to properly represent how they feel.
This also means that it may damage the data you receive.
If you’re not sure which type of question is best, ask two very similar questions, but each with a different scale. Then, when you test it before you launch, you’ll have a better understanding as to which question should be kept. When you compare the results, you will find that the results are comparable, if not identical.
Can a scale be too long?
The human memory has trouble with too many options, and many usability researchers talk about the 7 +/- 2 rule. This means that the optimum number of options to stop people from being overwhelmed is 7 (plus or minus 2). Theoretically, this means that the maximum number of options you should use is 9.
Realistically, if you agree with something, then deciding how many shades of agreement fits best can be tiring. Too many options can wear people out; and if you do that too often, your abandonment rates will rise.
Is a middle response necessary?
If you listen to your common sense, it might suggest to you that giving people a middle option allows them to avoid taking a real position. But this is not necessarily true. Some research has shown that even when forced to pick a side, middle-ranking respondents wouldn’t necessarily respond like the others would.
If you decide to eliminate a middle option, you may be forcing responses. When you do this, you introduce bias into the research, and run the risk of the research not being valid.
How should response options be labelled?
You will no doubt have seen all manner of option labels. Some people label only either end. Some people label every point. Some people only label the ends and middle. Then, there’s the question of whether or not to use just text, or just numbers, or some kind of blend.
So, what’s the right way to do this?
The best (read: Most accurate) surveys have labels that are clear, specific, and say exactly what that point means. When you remember that we want every respondent to be able to interpret the scale option in the same way without difficulty, this makes complete sense.
Scales that are partially labelled have been shown to be less effective than those that have a full set of labels.
A good comparison is a road with roadworks. A well-signed road gives you confidence about which lanes are merging, and where you need to be to avoid either collision or stand-still. But a poorly-signed set of roadworks leads to confused motorists, incomplete movement, and high levels of frustration.
This is what poorly labelled scale questions can be like. Your labels are the signposts that your respondents need to know exactly where they are, and exactly how it’s best for them to answer the question.
Some final tips
There are some other things you can do to make sure that your data is consistently high quality. As you practice labelling your scales, make sure that:
- Every option is labelled with words, so the meaning is clear
- Every option is relevant to the question
- Language is consistent throughout the whole survey
- Use scales that balance well (have the same number of options on each side)
- Consider providing a “Don’t know” option or “Prefer not to answer” option
By doing these things, and following the labelling guides above, you will consistently create scales that guide your respondents effectively.