What are validity and reliability?

One of the main functions of the research design is to ensure the highest possible quality of the data and this involves considering validity and reliability. The design of the research tools impacts significantly on the extent to which the data is valid and reliable. There has been extensive debate over the extent to which the terms validity and reliability can be applied within a qualitative context. Consequently alternative terms have been suggested by some qualitative researchers:

• credibility in place of validity

• dependability in place of reliability.

However Newby (2014) advocates the use of the terms validity and reliability within qualitative research, where a process of triangulation enables validity and reliability to be achieved. This Guide will also use the terms validity and reliability within qualitative contexts.

Understanding validity

In order to understand the possible factors influencing the validity of your findings, it is necessary to consider the different types of validity. To help you with this some of main types of validity have been explored in this MESHGuide (adapted from Cohen, et al., 2011).

Internal validity

This is concerned with the extent to which the data can be considered to support the conclusions which have been drawn from the data i.e. its plausibility and credibility.

In qualitative research internal validity can be improved through:

• reducing the degree of inference that has to be made by the subjects involved in the research

• triangulation through using more than one researcher to implement the research or to scrutinise the data collected

• using researchers who act as participants in the events being researched

• recording the data mechanically

External validity

This relates to the extent of the generalizability of the data to other contexts. External validity within qualitative research are influenced by:

• choice of sample e.g. if there are characteristics of the sample that are not common in the general population and which impact on the data collected in relation to the research focus

• the context-dependence of the findings i.e. if the findings have been influenced by factors operating within a particular context which were not related to the research focus e.g. factors related to the setting which are not operational in similar settings or events that have taken place in that setting in the past which are not characteristic of other similar settings.

This type of validity relates to the researcher’s understanding of factors to be researched within the project, which are termed constructs. This will influence the way in which the researcher operationalizes these constructs and it is this which will impact on the validity of the findings.

Ecological validity

The nature of qualitative research requires that the setting in which the research is carried out should be as natural as possible. This contrasts with quantitative research, which relies on variables being identified and controlled in order to be able to investigate the impact that different variables have on one another.

Two aspects need to be considered in relation to ecological validity:

• the approaches that need to be implemented to ensure that the research process is having as little impact as possible on the context being researched

• that the research process captures and describes as faithfully as possible the interplay of as many factors as possible that are impacting on the research focus (however this creates tensions in relation to ethical considerations such as ensuring anonymity of the subjects which need to be resolved).

The importance of ecological validity is illustrated in a project designed to establish the levels of anxiety the children were experiencing when engaged in science activities. The first pilot was carried out in a classroom where the researcher was unknown to the children. In her evaluation of the pilot the researcher identified this as a factor which was in itself influencing the children’s reactions within the activities and could even have impacted on their anxiety levels. This type of researcher effect has been termed the Hawthorne effect, where the research process is having an impact on the reactions or responses of the participants.

Understanding reliability

In qualitative research, reliability can be thought of in terms of the extent to which the data collected actually represents what is happening in the context being studied. Conversely in quantitative research the emphasis is on dependability, replicability and accuracy of the data. The table below compares reliability within qualitative and quantitative research contexts (Cohen, et al., 2011):