Measurement Scales and Response Formats

Overview

A response format is the structure through which an observation is recorded. It may be a category, ordered rating, count, measured quantity, date, free-text statement or position on a visual continuum. The format determines what distinctions respondents or observers are asked to make and what information is preserved for interpretation and analysis.

The familiar labels nominal, ordinal, interval and ratio describe increasingly strong interpretations of recorded values [1]. They are useful for thinking, but they should not be treated as permanent properties of a column. The meaning depends on the construct, measurement procedure and use. A temperature recorded in degrees Celsius supports equal-interval comparisons but has no meaningful zero for ratios; a symptom category may have a clear order without equal distance between categories.

Questionnaire designers make an additional set of choices: how many response options to offer, whether labels appear on all points, whether a neutral or “not applicable” option is needed, and which recall period should govern the answer. These choices can influence response error as much as the wording of the question itself [2–4,6,7].

Why this matters

An unsuitable response format can prevent the instrument from representing the intended construct. A yes/no item may hide meaningful frequency or severity. Too many categories may demand distinctions respondents cannot make. Poorly labelled options may be interpreted differently across participants. Combining “no”, “do not know” and “not applicable” into one category can create a variable with no coherent meaning.

Response format also shapes the analysis. Counts, ordered categories and continuous measurements require different summaries and models. The analysis should respect the information actually supported by the measurement rather than forcing every response into a convenient numeric code.

Levels of measurement

Nominal categories

Nominal values distinguish categories without implying order. Examples include clinic site, test type, blood group and recruitment channel. Numeric codes such as 1, 2 and 3 are labels; their arithmetic difference has no substantive meaning.

Binary variables are nominal variables with two categories, such as completed/not completed. A binary format is efficient when the distinction is natural and sufficient for the question. It is weak when the underlying phenomenon is graded or when uncertainty is common.

Ordinal categories

Ordinal values have a meaningful order, but the distance between adjacent categories is not guaranteed to be equal. Disease stage, functional class and responses from “strongly disagree” to “strongly agree” are common examples. The order should reflect one dimension. Categories such as “not difficult”, “difficult because of cost” and “difficult because of time” mix severity and reason, so they are not one ordinal scale.

Interval interpretation

An interval interpretation assumes equal differences across the scale. The difference between 10 and 20 represents the same amount of the construct as the difference between 30 and 40. Many constructed scores are analysed approximately this way, but the assumption should be supported by the measurement model and purpose rather than inferred merely because item responses were assigned numbers.

Ratio interpretation

A ratio scale has equal intervals and a meaningful zero representing absence of the quantity. Duration, distance, mass and count often support ratio interpretations. Ratios may still be misleading when zero is imposed by a detection limit or when the quantity is transformed.

Common response formats

Binary responses

Binary options suit clear states, such as whether consent was obtained or a documented event occurred. Include an uncertainty pathway when respondents may not know. Forcing “yes” or “no” can turn uncertainty into misclassification.

Unordered categories

Mutually exclusive categories may describe setting, route, diagnosis or reason. An “other, specify” option can reveal missing categories during development, but frequent use suggests the category set is incomplete.

Ordered verbal categories

Verbal categories represent intensity, frequency, agreement, confidence or difficulty. Options should be monotonic, non-overlapping and understandable. Labels such as “rarely” and “sometimes” may be interpreted differently unless a defined time frame or frequency accompanies them.

Likert-type agreement items

Likert-type items ask respondents to indicate agreement with statements [5]. Agreement is not a universal response task. It can invite acquiescence, mix truth with evaluation and perform poorly when the construct is frequency or severity. Direct questions often produce a clearer mapping: “During the past four weeks, how difficult was arranging an appointment?”

A multi-item Likert scale is conceptually different from one Likert-type item. Scores should be combined only when the items are intended to represent a coherent construct and the scoring model is justified.

Numeric rating scales

Numeric scales, such as 0 to 10, are compact and familiar in some settings. Anchors must define both endpoints and ideally the direction of higher scores. Intermediate numbers may not have equal meaning to every respondent.

Visual analogue scales

A visual analogue scale asks the respondent to mark a position on a line between anchored endpoints. It can offer fine resolution but may be difficult for electronic implementation, participants with visual or motor limitations, and settings where exact measurement of the mark is inconsistent.

Counts and frequencies

Counts preserve information about the number of events, such as appointment attempts or hospital admissions. The observation period must be explicit. A count of three has different meaning over one week and one year. Open-ended counts can produce heaping, extreme values or recall problems; grouped categories reduce burden but lose detail.

Dates, durations and measured quantities

Dates and times allow derivation of age, delay and follow-up duration. Collect the most direct and reliable element where possible. Asking “How many weeks postpartum were you tested?” may be less accurate than recording the test date and delivery date from sources and deriving the interval.

Comparing response formats for the same topic

illustrative

Table 1. Four ways to ask about appointment difficulty

Format	Example	Main advantage	Main limitation
Binary	Was arranging the appointment difficult? yes/no	brief	loses severity and uncertainty
Ordered verbal	not at all/a little/moderately/very/extremely difficult	interpretable anchors	category distances may differ
Numeric	0 = not difficult to 10 = extremely difficult	fine grading	numbers may be used inconsistently
Open text	What made arranging the appointment difficult?	reveals unanticipated issues	requires qualitative coding and is not a severity score

Choosing the number of response options

More categories can increase discrimination only when respondents can use them reliably. Too few categories compress variation; too many create artificial precision. The optimal number depends on the construct, population, administration mode and intended scoring. Five to seven verbal or numeric categories are common, but this is not a universal rule [3,4].

Labels should be:

• exhaustive enough to cover plausible responses;
• mutually exclusive;
• ordered on one dimension when an ordinal scale is intended;
• balanced around any neutral point;
• specific to a recall period; and
• understandable without interviewer reinterpretation.

DECISION GUIDE

Select a response format

1. Is the target a state, category, order, count, duration or continuous quantity?
2. Can respondents make the requested distinctions consistently?
3. Does the format match the intended recall period and mode of administration?
4. Are uncertainty, non-applicability and refusal conceptually distinct?
5. Will the planned scoring preserve the information needed for the research question?

Neutral, uncertain and non-applicable responses

A neutral attitude is not the same as not knowing. “Not applicable” means the question does not logically apply. “Prefer not to answer” reflects non-response. These categories should not be placed automatically at the midpoint of a scale or included in a score as though they represent a substantive level.

Researchers sometimes remove “do not know” to discourage satisficing. This can instead force guesses. The decision should follow the target construct: knowledge questions may legitimately have an “I do not know” response, whereas an immediate pain rating usually does not require one.

Recall periods and reference frames

A response format is incomplete without a reference period when the construct varies over time. “Usually”, “recently” and “often” invite different interpretations. A fixed period such as the past seven days improves standardisation but may not represent longer-term experience. The period should match the research question and the frequency of the event.

Questions should also specify the object and context. Confidence in arranging testing may differ between the participant’s usual clinic and a hospital appointment. A vague item can average incompatible situations.

Scoring and missing responses

The protocol should state whether items are analysed individually or combined, which direction indicates more of the construct, how reverse-worded items are handled, and how missing items affect score calculation. A sum score assumes that the selected items and scoring rule form a meaningful scale. Numeric coding alone does not justify addition.

Reverse wording is sometimes used to reduce response patterns, but it can introduce confusion and method effects. Clear direct wording with careful ordering is often preferable.

PROTOCOL CHECKLIST

Document response-format decisions

• exact item wording and recall period;
• response options and endpoint anchors;
• rationale for the number and order of categories;
• treatment of neutral, unknown, not-applicable and refusal responses;
• administration mode and any visual layout requirements;
• coding and score direction;
• handling of missing item responses; and
• planned analysis that respects the supported measurement level.

Interface with statistical analysis

The recorded format influences permissible summaries but does not dictate one automatic test. Nominal variables are summarised by counts and proportions. Ordinal responses require methods that preserve order when the order is substantively important. Counts often have skewed distributions and may require count models. Continuous quantities should generally remain continuous unless a clinically defensible threshold is central to the question.

A total score derived from ordered items may sometimes be analysed as approximately continuous, but the justification should consider dimensionality, score distribution, number of items, measurement model and intended inference. The Biostatistics section provides detailed guidance for categorical, ordinal and numerical outcomes.

Item response models and category functioningMMed extension

Item response theory and Rasch-family models evaluate how item responses relate to an underlying latent trait. They can identify disordered thresholds, poorly functioning categories and items that behave differently across groups. These models do not rescue weak content: a statistically elegant item bank still fails when important aspects of the construct were never included.

Worked example

WORKED EXAMPLE

Redesigning an ambiguous frequency item

Draft item: “I often had difficulty attending the clinic.” Responses: strongly disagree to strongly agree.

Problems: The item mixes frequency with difficulty, has no time period and uses agreement for a behavioural experience. Participants may agree because one severe episode occurred or disagree because difficulties were infrequent.

Revised item: “During the first 12 weeks after delivery, on how many occasions did practical problems prevent you from attending a planned health appointment?” Responses: none/once/twice/three or more times/not sure.

Reasoning: The revision specifies a period, event and consequence. It produces an ordered frequency category rather than an unclear agreement score. A separate item can ask how difficult arranging an appointment felt.

Suggested protocol or reporting wording

Common mistakes

Knowledge check

Why is a four-category response coded 1 to 4 still ordinal?

Answer and reasoning

The codes preserve the order of categories but do not establish equal differences between them. The substantive interpretation comes from the response definitions and measurement model, not from the assigned numbers.

When is a yes/no response most defensible?

Answer and reasoning

It is defensible when the target is a clear binary state and uncertainty is uncommon or handled separately. It is less suitable for graded constructs such as severity, confidence or frequency.

Should a 'not applicable' response be scored as the lowest burden category?

Answer and reasoning

Usually not. Non-applicability means the item does not describe the participant’s situation, whereas the lowest burden category is a substantive response. Combining them changes the score’s meaning and may create bias.

What is the main problem with asking whether a participant 'often' experienced a barrier?

Answer and reasoning

“Often” has no standard frequency and may be interpreted relative to different periods or opportunities. A defined time frame and event count or fully anchored frequency categories improve comparability.

Key take-home messages

SUMMARY

Key take-home messages

• Response formats should match the construct and the distinctions respondents can make.
• Nominal, ordinal, interval and ratio interpretations concern meaning, not merely numeric coding.
• Neutral, unknown, non-applicable and refusal responses are conceptually different.
• Recall periods, endpoint labels and score direction are part of the measurement instrument.
• Preserve useful information unless a scientifically justified category or threshold is central to the question.

References

1. Stevens SS. On the theory of scales of measurement. Science. 1946;103(2684):677–680. https://doi.org/10.1126/science.103.2684.677
2. Tourangeau R, Rips LJ, Rasinski K. The psychology of survey response. Cambridge: Cambridge University Press; 2000. https://doi.org/10.1017/CBO9780511819322
3. Krosnick JA, Presser S. Question and questionnaire design. In: Marsden PV, Wright JD, editors. Handbook of survey research. 2nd ed. Bingley: Emerald; 2010. p. 263–313.
4. Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford: Oxford University Press; 2015. https://doi.org/10.1093/med/9780199685219.001.0001
5. Likert R. A technique for the measurement of attitudes. Arch Psychol. 1932;140:1–55.
6. US Food and Drug Administration. Patient-focused drug development: methods to identify what is important to patients. Silver Spring: FDA; 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-focused-drug-development-methods-identify-what-important-patients
7. Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method. 4th ed. Hoboken: Wiley; 2014.