Qualitative research (oud)

Please note that this guideline is currently being updated. The updated guideline will be available soon. For more information, please contact aph.sqc@vumc.nl

shutterstock_230270845

Aim

To give researchers a short description of the field of qualitative research and provide them with relevant literature.

 

Requirements

  • Clear documentation in the research protocol, field notes and logbook about the choice of research method(s), the selection of respondents, quality checks, the role of the researcher in the research process, the analysis and strategy for publication;
  • To ensure the respondents’ anonymity.

 

Documentation

Research protocol: state the research question, the research design and data handling (collection, entry, storage, analysis);
Field notes: keep a diary during the period of data collection and analysis, e.g. regarding the selection of respondents, the role of the researcher in the data collection process, the quality criteria (credibility, transferability, dependability, confirmability; see below);
Logbook: decisions and arguments regarding important changes in design, methodology, research instruments and analysis plan.

 

Responsibilities

The allocation of responsibilities depend on agreements in the project team. In general:
Executing researcher: To carry out the research as described in the protocol and keep detailed field notes.
Project leaders:
  • To supervise the executing researcher;
  • To discuss changes or differences in relation to the research protocol together;
  • To make sure the executing researcher is capable to carry out qualitative research or to provide him or her with (extra) training.
Research assistant: To assist in data collection, data entry and data analysis.

 

How To

1. Introduction
Qualitative research methods are increasingly being used within medical research; a trend that is also being observed within the EMGO+ Institute. This guideline contains a short description of some important topics of and features in qualitative research.
For a more comprehensive checklist describing aspects of designing and implementing qualitative research, see the AMC guideline [1] Richtlijnen Kwalitatief Onderzoek_AmCOGG (in Dutch). This document gives a description of the following topics:
  • 1. Research question
    • a. Problem
    • b. Question
  • 2. Research design
    • a. Population
    • b. Methods
    • c. Instruments
  • 3. Data
    • a. Data collection
    • b. Data acquisition
    • c. Data entry
    • d. Data cleaning
    • e. Data storage
    • f. Data analysis
  • 4. Reporting
    • a. Scientific publications
    • b. Presentations
    • c. Non-scientific publications
  • 5. Other

 

2. Types of research questions
Qualitative research methods are particularly appropriate if:
– You don’t know very much about a topic (exploratory research);
– You are interested in the nature of a phenomenon;
– You are interested in the respondents’ experiences or perceptions;
– You want to study the meaning of interactions, processes, behaviors, feelings, attitudes and experiences;
– The outcome of a study requires a more in-depth explanation.

 

3. Important features of qualitative research
– Iterative process
Data collection and analysis take place simultaneously and the researcher can also go back to the previous phase. As a consequence, the analysis of the material starts as soon as the first data have been collected. In a way, this analysis guides subsequent data collection. The researcher starts off with a few global insights 1) that provide an initial directional guide and 2) that are adapted to the situation in the field of study in the course of the research process. Having first become “openly” acquainted with the field, insights are provided into the important focus points for the research questions, and finally focused work can be undertaken using the specific research questions.
– Theoretical saturation
The recruitment of respondents/patients in qualitative research is not aimed at generalization. Instead, researchers aim for theoretical saturation: “inclusion” continues until a point is reached where there is a sense that sufficient insight has been acquired about the research phenomenon, i.e. until the researcher(s) sense that no new information is provided by new participants. Important principles in this process are constant comparison, purposive sampling, and theoretical sampling: the researcher is constantly looking for new “cases” that are able to sharpen up, confirm or correct earlier cases [2]. This results in the research group usually being relatively small (maximum of tens of participants) and heterogeneous instead of large group size and (particularly for subgroups) homogeneous, as is often the case in quantitative research.

 

4. Methods in qualitative research
Three fundamental methods can be identified for qualitative research, which vary in implementation. One can use a single method or combine various methods (methodological triangulation).
Observation
Observation is especially suitable if you want to know how people behave in practice. This can be done by (intensively) participating in the field you wish to study (participatory observation/emphasis on participation), or by being present in the field and making observations (observational participation/emphasis on observation).
Interviewing
You can use interviews if you are interested in opinions, experiences or attitudes. Various types of interviewing can be used, depending on the research question and level of exploration. A distinction is made here between:
– The level of interview structure: Open interview/in-depth interview, semi-structured interview, structured interview. Often a list of themes (topic list) is used, which can be followed more or less strict.
– Single or repeated interviews
– Individual or group interviews (focus groups): Interaction between participants is central to focus groups. Participants are stimulated by others’ ideas and will make up their own opinion, or modify their opinion having listened to others [3]. As participants are able to ask each other questions and explain themselves to others, a focus group can produce more than the sum of the individual interviews.
Document analysis
This may involve personal as well as non-personal documents (e.g. letters, diaries, photos, archive, reports of meetings, newspaper articles, film, drawings, etc.). Documents are used, in particular, for historical research.

 

5. Quality procedures
Checks on the quality of the data can be assessed in a number of different ways. Two important principles are the so-called “member check” and “peer debriefing”. Applying a member check means that the researcher checks his interpretation with the participants, for instance by restating or summarizing information and then question the participant to determine accuracy or by sharing all the findings with participants at the end of the data collection. During peer debriefing a researcher checks his interpretations with fellow researchers.

 

Furthermore, Devers [4] formulated several strategies for enhancing the accuracy of qualitative research:
Criteria
Strategies
Credibility / Internal validity
– Triangulation. The purpose of triangulation is to make use of multiple data sources, investigators, methods or theories as much as possible to provide corroborating evidence.
– Search for disconfirming evidence (“deviant” or “negative” cases). Instead of ignoring cases or information that “doesn’t fit”, the researcher actively looks for cases that do not fit the pattern and refines the theory and working hypotheses in light of this evidence. The researcher(s) continues this process until all cases fit, eliminating all outliers and exceptions.
– Subject review (Also called “member checking” and “dialogue with participants”). The researcher(s) obtains for research “subject”, group member, or participant views of the credibility of interpretation and findings. In some cases, this strategy is also used to increase the probability that research results will be used.
Transferability / External validity
– Detailed description of the study context, the investigator’s role in the context and how the context affects the ability to answer the original research question.
Dependability / Reliability
– Data archiving/creating an audit trail. The researcher(s) should ensure the completeness and accuracy of documents (e.g. interviews, observations, etc.) and be clear about the coding schemes and data analysis process. Theoretically, this would allow someone not connected with the study to review the primary documents and coding schemes to assess whether the findings, interpretations, and conclusions are supported.
– Skeptical peer review. A skeptical peer-reviewer plays the role of devil’s advocate, asking difficult questions about methods, meanings, and interpretation of the data. This process provides an external check on the research.
Confirmability / Objectivity
– Triangulation. See description above.
– Skeptical peer review or audits. See above.
– Search for disconfirming evidence or negative cases. See above.
– Reflective journal keeping by the researcher. Because the researcher is the research instrument in qualitative research, he or she should keep journal notes on how his or her personal characteristics, feelings, and biases may influence the work and how he or she tries to manage them to the extent possible.

 

6. Data analysis
Data analysis can be inductive (i.e. solely based on the data, like in grounded theory [5]), deductive (starting from predetermined themes or categories based on a theory or the literature), or somewhere in between. This depends on the aims of the research [1].
There are various software programmes that support the analysis of qualitative data, such as Atlas.ti, MaxQDA, NVivo and KWALITAN. These programmes can be a useful tool in ordering the data, but you will need to order the data yourself.
The Atlas.ti programme has been acquired for analysis in the EMGO+ Institute in August 2006. Consult the data manager Wim Kraan for license information.

 

7. Publication
Qualitative research is published more and more frequently within medical science. Certain criteria are imposed on publishing qualitative data. For instance, the British Medical Journal has created a checklist with its own set of criteria for reporting qualitative research:
Qualitative research checklist BMJ
– Was the research question clearly defined?
– Overall, did the researcher make explicit in the account the theoretical framework and methods used at every stage or the research?
– Was the context clearly described?
– Was the sampling strategy clearly described and justified?
– Was the sampling strategy theoretically comprehensive to ensure the generalizability of the conceptual analysis (diverse range of individuals and settings, for example)?
– How was the fieldwork undertaken? Was it described in detail?
– Could the evidence (fieldwork notes, interview transcripts, recordings, documentary analysis, etc.) could be inspected independently by others: if relevant, could the process of transcription be independently inspected?
– Were the procedures for data analysis clearly described and theoretically justified? Did they relate to the original research questions? How were themes and concepts identified from the data?
– Was the analysis repeated by more than one researcher to ensure reliability?
– Did the investigator make use of quantitative evidence to test qualitative conclusions where appropriate?
– Did the investigator give evidence of seeking out observations that might have contradicted or modified the analysis?
– Was sufficient of the original evidence presented systematically in the written account to satisfy the skeptical reader of the relation between the interpretation and the evidence (for example, were quotations numbered and sources given)?
This checklist is also useful if you want to publish in another journal.

 

Appendices/references/links

[1] Netwerk Kwalitatief Onderzoek AMC – UvA. Richtlijnen voor kwaliteitsborging in gezondheids(zorg)onderzoek: Kwalitatief onderzoek. Amsterdam, 2002. Richtlijnen Kwalitatief Onderzoek_AmCOGG
[2] Wester, F. Strategieën voor kwalitatief onderzoek (Strategies for qualitative research). Coutinho, Muiderberg, 1987. ISBN: 90-6283-896-0.
[3] Assema P van, Mesters I, Kok G. Het focusgroep-interview: een stappenplan (Focus group interviews: A stepwise approach). TSG 1992;70:431-437
[4] Devers KJ. How Will We Know “Good” Qualitative Research When We See It? Beginning the Dialogue in Health Services Research. Health Services Research 1999; 34 (5): 1153-1188.
[5] Strauss A, Corbin J. (eds.) Grounded Theory in Practice. London: Sage 1997.

 

Basic books on qualitative research:
– D.B. Baarda, M.P.M. de Goede, J. Teunissen. Basisboek kwalitatief onderzoek. Praktische handleiding voor het opzetten en uitvoeren van kwalitatief onderzoek (Basic qualitative research: Practical guide for designing and conducting qualitative research). Wolters-Noordhoff bv Groningen/Houten, 2001. ISBN: 90 207 2485 1
– Hennie Boeije. Analyseren in kwalitatief onderzoek. Denken en doen (Analysing qualitative research: Thinking & doing). Boom onderwijs, 2005. ISBN:90 8506078 8
Books/articles (specific to) qualitative research methods:
– Gubrium JF, Holstein JA (eds). Handbook of Interview Research. London: Sage 2002.
– Pope C, Mays N. (eds) Qualitative Research in Health Care. Second edition. BMJ books 2000. ISBN: 0-7279-1396-4 (A collection of BMJ articles on qualitative research in general and specific data collection methods).
– Yin R.K. Case study research. Design and methods London: Sage 1989.

 

Series of 4 articles about qualitative research methods in Huisarts & Wetenschap (General Practitioners & Science):
– Philipsen H, Vernooy-Dassen M. Kwalitatief onderzoek: nuttig, onmisbaar en uitdagend (Qualitative research: Useful, essential and challenging). Huisarts & Wetenschap (General Practitioner & Science) 2004;47(10): 454-457
– Hak T. Waarnemingsmethoden in kwalitatief onderzoek (Observations in qualitative research). Huisarts & Wetenschap (General Practitioner & Science) 2004;47(11): 502-508
– Wester F. Analyse van kwalitatief onderzoeksmateriaal (Analysing qualitative research material). Huisarts & Wetenschap (General Practitioner & Science) 2004;47(12): 565-570
– Zwieten M van, Willems D. Waardering van kwalitatief onderzoek (Evaluating qualitative research). Huisarts & Wetenschap (General Practitioner & Science) 2004;47(13): 631-635

 

Website
– www.kwalon.nl
The KWALON platform is an association that aims to develop and disseminate qualitative methods in behavioural and social science research.

 

Audit questions

  1. Which research method(s) have been used? Has triangulation been used?
  2. How have respondents been selected?
  3. How was theoretical saturation employed? Has purposive sampling been used? If so, please provide an accurate description of the procedure.
  4. Have there been quality checks on the data through a member check or peer debriefing?
  5. Has the patients/respondents’ anonymity been safeguarded? If so, how?
  6. How has the information obtained been recorded? Have various memos been used?
  7. Has the role of the researcher in the data collection process been determined and described? If so, how?
  8. How have the available data been analyzed?
  9. Describe the strategy for the publication of results. Have or will the BMJ guidelines been used?

V2.0: 12 May 2015: Revision format
V1.2: 1 Dec 2011: Removal of link kwalitatief sterk
V1.1: 1 Jan 2010: English translation
V1.0: 23 Nov 2006: Draft version has been rewritten in full