Selecting measurement instruments
In the majority of research problems, multiple existing measurement instruments are available to use. In this case it is important to assess the quality of the existing instruments and to select the most suitable measurement instrument for your study. This chapter provides the most important steps in selecting measurement instruments, alongside references to helpful literature.
To select the measurement instrument(s) that are most suitable to the research question(s)
Step 1: Define the construct and conceptual model
The first step is to decide what you want to measure in your study, this is called the construct (synonyms: outcome, concept, domain, etc.). The construct should be clearly defined in order to select a suitable measurement instrument. Conceptual models can help in clearly defining your construct, to get insight in the potential levels and subdomains of the construct, and to differentiate between other related and unrelated constructs. Examples of conceptual models are the International Classification of Functioning (ICF), the model by Wilson and Cleary (1), OMERACT Filter 2.0 (2), or the model of the Dutch Flemish PROMIS.
Step 2: Select the type of measurement instrument
The selection of the type of measurement instrument depends on the construct of interest, the research question, and purpose of the study. There are different types of measurement instruments:
- Patient-reported outcome measures (PROM): PROMs are either questionnaires or interviews that capture the patients perspective in relation to a health condition.
- Patient-reported experience measures (PREM): PREMs are either questionnaires or interviews that capture the patients experiences with care.
- Clinician-reported outcomes: an assessment from a trained healthcare professional, based on their observation and clinical judgement.
- Observer-reported outcomes/ proxy-reported outcomes: for outcomes that cannot be directly measured, the outcome can be based on the assessment of someone other than the patient or clinician. For example, a report by a parent or caregiver (in case or a child or cognitively impaired patient respectively).
- Performance-based outcomes: these outcomes are based on a task for the patient. For example, the 40-steps test.
- Lab tests: for example a blood test.
For example, when you want to measure physical functioning, you could be interested in what people perceive they can do in their own invironmentor what they actually can do (e.g. under supervision of a physiotherapist in a standardized setting). Therefore, depending on how the construct is defined, the type of measurement instrument is selected.
At the website https://www.comet-initiative.org/ you can check whether a core outcome set (COS) is developed for your specific target population. A COS is a consensus-based agreed minimum set of outcomes that should be measured and reported in all clinical trials of a specific disease or trial population (3). If such a COS exist, we recommend you to measure all the outcomes proposed in the COS.
Step 3: select the measurement instrument
When the construct is clearly defined and the type of measurement instrument is selected, an instrument can be selected. In this selection process, a couple of steps are important:
- The aim of the measurement: the aim of the measurement refers to which construct you want to measure in which target population, but also to the purpose of measurement. For example, you want to use the measurement as an outcome in a trial or clinical practice, or you want to diagnose patients with a specific condition, or you want to screen people on specific conditions. The purpose of measurement determine which measurement properties are most relevant. In diagnostics the reliability and validity are most important, whereas in outcomes assessment content validity, and responsiveness are crucial. When instruments are used to detect very small changes or on individual level, the measurement error of the instrument needs to be small. Internal consistency is only relevant for multi-item (sub)scales measuring (uni-dimensional) constructs that are based on a reflective model.
- The quality of measurement properties: when selecting a measurement instrument, its measurement properties should be assessed and taken into consideration. Three domains of measurement properties can be distinguished: reliability (is the instrument free of measurement error?), validity (does the instrument measure what it wants to measure?) and responsiveness (the ability of the instrument to detect changes in the construct of interest). To find information on the nine measurement properties of a specific instrument for your target population, you could search for systematic reviews on measurement instruments in the COSMIN database of systematic reviews. When no systematic reviews on your construct and/or for your target population are available, you can identify all relevant studies within the literature, with the COSMIN search filters (see the user manual of the COSMIN methodology for conducting systematic reviews for an explanation).
- The feasibility: issues as costs, time, setting, and respondent burden also play a role in the selection of the instrument.
Examples of instruments
Though your choice of instrument is dependent on the steps mentioned earlier, there are some widely used initiatives, that will be listed here:
- PROMIS (patient-reported outcomes measurement information system): provides PROMs for many different constructs (physical, mental and social health). These are generic (non-disease specific) PROMs, available for children and adults, in different languages. PROMIS offers short forms and CATs (computerized adaptive testing). PROMIS has an international website and a Dutch website.
- ICHOM: provides standard sets (COS) with outcomes that matter most to patients for many different conditions or specific patient populations. Their standard sets elaborate on the core principles of Michael Porters value based healthcare (VBHC) (4). The ICHOM standard sets include a list of outcomes, the recommended measurement instruments, and a timeline for measurement points.
- Meetinstrumenten Home – Meetinstrumenten in de zorg provides an overview of over 400 PROMs, observation instruments and performance-based outcomes, in Dutch. The PROQOLID database provides measurement instruments in English.