Among the many threats to internal validity, we consider the following:
- History
During the time that a research is taking place, some events may occur that confuse the relationship being studied. These are events may either increase or decrease the expected outcomes of the project. These are events which are not part of the project and they are not planned for. They may just happen in the process of the research and have tremendous effects
on the results of the study. - Testing
The process of taking a test can affect the scores of a second test. The mere experience of taking the first test can have a learning effect that influences the results of the second test. Subjects who are given a pretest are likely to remember some of the questions or some of the errors they made when they are taking the posttest. They are also likely to do somewhat better 0n the posttest than they did on the pretest. - Instrumentation
This threat to internal validity results from changes between observations, in measuring instruments or in observer. Using different questions at each measurement is an obvious source of potential trouble, but using different observers or interviewers also threatens validity. Observer experience, boredom, fatigue, and anticipation of results can all distort the results of
separate observations. For example, an experienced interviewer may obtain more complete information from a correspondent than an inexperienced interviewer. The additional information may be due to the fact that the interviewer has become more skilled in asking questions or observing events and not due to the effect of the program or observing the effects of
the treatment. - Maturation:
Changes may also occur within the subject that is a function of the passage of time and not specific to any particular event. These are of special concern when the study covers a long time, but they may also be factors in tests that are as short as an hour or two. A subject can become hungry, bored or tired in a short time, and this condition can affect response results. - Selection
An important threat to internal validity is the differential selection of subjects for experimental and control groups. Validity considerations require that groups be equivalent in every respect. If subjects are randomly assigned to experimental and control groups, this selection problem can be largely overcome. Additionally, matching the members of the groups on key factors can
enhance the equivalence of the groups. Validity considerations require that the groups be largely overcome. Additionally, matching the members of the groups on key factors can enhance the equivalence of the groups. - Experiment Mortality
This occurs when the composition of the study groups changes during the test. Attrition is especially likely in the experimental group, and with each dropout, the group changes. Because members of the control group are not affected by the testing situation, they are less likely to withdraw. In a compensation incentive study, some employees might not like the change in compensation method and withdraw from the test group; this action could distort the comparison with the control group that has continued working under the established system, perhaps without knowing a test is under way.
Factors affecting the external validity of the study
Internal validity factors cause confusion about whether the experimental treatment (x) or extraneous factors are the source of observation differences. In contrast, external validity is concerned with the interactions of the experimental treatment with other factors and the resulting impact on abilities to generalize to times, settings or persons.
- Reactive effects of testing:
If pre-testing has been used and which sensitizes the experimental subjects to the particular treatment, then the effect of the treatment may be partially the result of the sensitization of the pre-test. - Interaction effects of selection bias
If the samples draw from the study is not representative of the larger population, then it would be difficult to generalize findings from the samples to the population, and this may arise when the samples are not drawn randomly from the population. Consider a study in which you a cross-section of a population to participate in an experiment, but a substantial number refuse. If you do the experiment only with those who agree to participate, can the results be generalized to the total population? - Other reactive factors
The experimental settings themselves may have a biasing effect on a subject’s response to the treatment. An artificial setting can obviously give results that are not representative of large populations. Suppose workers who are given an incentive pay are
moved to a different work area to separate them from the control group. These new conditions alone could create a strong reaction condition.
If subjects know they are participating in an experiment, there may be a tendency to roleplay in a way that distorts the effect of the experimental treatment.
Common effects related to the research process
There are other situations in which the internal and external validity of the study may both be threatened simultaneously. This is brought about by what we call research effects, which have nothing to do with the treatment.
1. Hawthorne Effect
This refers to a situation where subject awareness of being in an experimental group motivates them to perform better. Therefore the most influential factor on the subjects is not the independent variable but their awareness of being in a special group.
2. The placebo Effect
This is common to medical studies. Researchers observe that a drug administered to any group of parties has two effects.
- Chemical effect
- Psychological effect
To counteract this effect, researchers use a placebo and this is an inactive substance which has the same colour and tests as the active drug; Half of the subjects (experimental group) are given the active drug and the other alf (control group) are given the placebo inactive drug. If there is a significance difference between those two groups, the drug may be said to have a significance effect.
