Quantitative research methods
ATL: Essential understandings
Psychologists do not "prove" anything; they either support or refute a hypothesis.
A researcher will choose the method or methods that are most suitable for a specific research study.
Quantitative research methods emphasize objective measurements and the statistical analysis of data.
Variables must be fully operationalized in order for a study to have validity.
Extraneous variables may affect the validity of a study.
One of the most widely used methods in the study of behaviour has been the experiment. The goal of an experiment is to determine whether a cause-and-effect relationship exists between two variables. The experiment is an example of quantitative research, which generates numerical data. These data can be statistically tested for significance in order to rule out the role of chance in the results.
Let us say that a researcher wants to find out if noise affects one’s ability to recall information. The aim of the study is to see if one variable (noise) has an effect on another variable (recall of information). The variable that causes a change in the other variable is called the independent variable (IV). This is the variable that the researcher deliberately manipulates. The variable that is measured after the manipulation of the independent variable is called the dependent variable (DV).
A key characteristic of an experiment is the use of controls. The idea of "control" is that when the researcher manipulates the independent variable, all other possible variables stay the same. In other words, the procedure must be exactly the same in both groups; the only difference should be the manipulation of the independent variable.
For example, if we are testing the role of noise in one's ability to recall a list of words, one group would read the list while listening to music. Another group would read the list in silence. Otherwise, there should be no other difference between the groups. Some examples of controls for this study would include:
- The list of words would be the same - the same words, the same font, the same size font, the same order of the words.
- The conditions of the room should be the same. If one room has a lot of posters on the wall with information, while the other room has bare walls, this could theoretically influence the results.
- The temperature of the rooms should be the same.
- The time of day when the test is taken should be the same.
Both the independent and dependent variables must be operationalized. In other words, they need to be written in such a way that it is clear what is being measured. In the example used above, noise is the independent variable. This could be operationalized as dissonant rock music played at a volume of 100 decibels. An operationalized dependent variable could be the number of words remembered from a list of 30 words. Now we know exactly what the IV is and what you are going to measure in order to support your research question. Simply stating that your dependent variable is "the results of the study" is not enough as it does not say anything about what is actually being measured.
Another characteristic of experiments is that they are highly standardized. This means that they have procedures that are written in enough detail that they can be easily replicated by another researcher.
Finally, a true experiment randomly allocates participants to conditions. With random allocation, participants have the same chance of being assigned to the experimental or the control condition. This lessens the potential for characteristics of the individuals influencing the results.
ATL: Be a thinker
Operationalizing your variables is one of the most important steps to setting up an experiment. For each of the studies below, think about what you would want to measure.
Experiment 1: A study of stress on one's level of aggressive behaviour.
- The independent variable (IV) is stress. How could you change the level of stress in the different conditions? (Remember, this must meet ethical standards).
- The dependent variable (DV) is level of aggression. What would you actually measure?
Experiment 2: A study on how one's level of self-esteem affects their problem-solving skills.
- The independent variable (IV) is level of self-esteem. How could you change the level of self-esteem in the different conditions?
- The dependent variable (DV) is problem-solving skills. What would you actually measure?
Before carrying out an experiment, researchers first read through the research which is available on the research question that interests them. Their research question is the aim of the study, usually limited to a certain population. For example, the aim of this study is to determine whether listening to dissonant rock music while studying vocabulary words affects recall in adolescent girls. After the aim of the study is decided, the researcher formulates a hypothesis. The hypothesis is a prediction of how the independent variable affects the dependent variable.
An experimental hypothesis predicts the relationship between the IV and the DV - that is, what we expect will come out of the manipulation of the independent variable. In this case, we will have two conditions: one condition where participants have to recall words with loud music, and one where the participants recall words with no music. In the second condition, there is no noise. This is called the control condition, because we compare the two conditions—that is, one with noise and one with no noise—in order to see if there is a difference.
An example of an experimental hypothesis could be: Listening to dissonant rock music played at 100 decibels will decrease the number of words that adolescent girls are able to recall from a list of 30 words. In an experimental hypothesis, the IV (listening to dissonant rock music played at 100 decibels) is predicted to have an effect on the DV (the number of words recalled from a list of 30 words).
In experimental research, it is conventional to formulate both a null hypothesis and an experimental hypothesis. The null hypothesis states that the IV will have no effect on the DV, or that any change in the DV will be due to chance.
An example of a null hypothesis could be: Listening to dissonant rock music played at 100 decibels will have no significant effect on adolescent girls’ ability to recall words from a list of 30 words; any change in the individual’s ability to recall a list of words is due to chance.
You may find it strange to make a null hypothesis, but in fact, it makes sense. The researcher wants to reject the null hypothesis to show that the predicted cause-and-effect relationship between the IV and the DV actually exists. Sometimes, however, we have to accept the null hypothesis. This would happen if the results showed that there was no relationship between music and the recall of words. It is important to recognize that psychologists never prove anything - they can only disprove. Our goal is either to accept the null hypothesis, which means that we have found that there is no relationship between two variables, or reject the null hypothesis, which means that there is some type of relationship between the two variables.
ATL: Be a researcher
Each of the following statements is a research proposal. For each of the statement, change it into an experimental and null hypothesis. Be sure that you have operationalized your variables.
- Does the presence of other people change people's risk-taking behaviour?
- I am interested in finding out if eating high levels of carbohydrates before an exam affects your success on the exam.
- A teacher wonders if students would do better in IB psychology if the class were more visual.
The advantage of a laboratory experiment is that it allows the researcher to control for extraneous variables - this is, other variables that may influence the results of the study - in this case, the behaviour or cognitive processes of a participant. But as we know from our study of validity, if we control a situation in an experiment, it may not reflect what happens in real life. In that case, the study may have low ecological validity.
When we do studies outside of the laboratory, in the "real world", this is called a field experiment. Field experiments have two key limitations - they cannot control for extraneous variables and they cannot be easily replicated.
There are also ethical considerations with field experiments. Let's say that you want to do a study of helping behaviour on a city's metro (subway). You decide that you are going to have someone faint on the metro to see if anyone will help. In one condition, the person in need of help is well dressed, wearing a suit and tie. In the second condition, he is wearing torn jeans and a t-shirt advertising a hard rock band.
The concern is that in field experiments it is often not practical to get informed consent. The general rule is that people may be observed in public spaces where they would expect to be observed by others. The other ethical concern is debriefing. Often the nature of field experiments makes it impossible to explain to the unwary participants what has just happened.
Think about the study scenario described above. This is similar to a famous study done by Piliavin, Rodin and Piliavin (1969) which we learn about in the human relationships unit. The passengers were not willing participants in the study. And when the door to the train opened, they disembarked and were not debriefed. How might it have affected passengers to think that they had seen someone pass out on the metro - potentially from a heart attack or some other terrible problem - and yet they didn't help? Just like with laboratory experiments, it is important that an ethics board approves any field experiments in order to guarantee the protection of participants.
When discussing public spaces, perhaps no place is more controversial than public toilets
Middlemist, Knowles & Matter (1976) designed an experiment to test how the presence of others affected the time it took men to urinate in a public toilet.
The researchers did their field experiment in a public toilet where there were three urinals. One of the researchers stood either at the urinal right next to the unsuspecting participant, one away from the participant, or was not present.
The other researcher was hidden in the stall next to the urinal. His job was to measure the time it took for the participant to begin urinating.
The results showed that with no one present, the average onset of urination was 4.8 seconds, with a researcher present one urinal away, the onset was 6.2 seconds. Finally, with the researcher in the next urinal, it took 8.4 seconds.
1. Do you feel that this study meets ethical standards? Why or why not?
2. Does this study have any value? How do you think that we could apply the findings of this study?
Quasi and natural experiments
In both quasi and natural experiments, participants are not randomly allocated to conditions. In quasi-experiments, participants are grouped based on a trait or behaviour. For example, in a quasi-experiment you may have two groups attempt to memorize a list of words. One group is made up of people who have been diagnosed with depression and the other group, the control, would not have such a diagnosis. As the variable that we want to study - having clinical depression - cannot be randomly assigned, this study is a quasi-experiment. Other typical examples of IVs used in quasi-experiments include gender, culture and age.
A subset of quasi-experiments is "natural experiments." Although these two terms are often used interchangeably, a natural experiment usually refers to an independent variable that is environmental in nature and outside of the control of the researcher. Most natural experiments work on a pre-test, post-test design - that is, the behaviour is measured both before and after the variable was introduced. For example, in 2017 the Czech Republic in introducing a smoking ban in all bars, restaurants and workplaces. Researchers may decide to measure the level of clinical depression in the city of Prague before the ban is put in place and then again six months after the ban is in place. Once again, there is no random allocation to groups, but the researchers would be able to see if there is any difference in mental health after the change in the IV.
Quasi and natural experiments do not show direct causation, but they are able to imply a causal relationship between an IV and a DV.
In an experiment, researchers attempt to control as many variables as possible. However, this is not always easy. Extraneous variables (also called confounding variables) are undesirable variables that influence the relationship between the independent and dependent variables.
Demand characteristics occur when participants act differently simply because they know that they are in a study. They may try to guess the aims of the study and act accordingly.
There are several ways that participants may influence the experiment because they believe that they know what the researcher is looking for or what the researcher is trying to do. In qualitative research, demand characteristics are often called "participant expectations." These include, but are not limited to:
Expectancy effect: the participant attempts to discern the experimenter's hypotheses with the goal of "helping" the researcher. This may result in acting in a certain way or giving the "right answer."
Screw you effect: the participant attempts to discern the experimenter's hypotheses, but only in order to destroy the credibility of the study.
Social desirability effect: This is when the participant answers in a way that makes him/her look good to the researcher. This is done to avoid embarrassment or judgement.
Researcher bias is when the experimenter sees what he or she is looking for. In other words, the expectations of the researcher consciously or unconsciously affect the findings of the study. Using a double-blind control can help to avoid this. In this design, not only do the participants not know whether they are in the experimental or control group, but the person carrying out the experiment does not know the aim of the study, nor which group is the treatment and which one the control group.
Participant variability is a limitation of a study when characteristics of the sample affect the dependent variable. This can be controlled for by selecting a random sample and randomly allocating the participants to the treatment and control groups.
One other consideration is artificiality. This is when the situation created is so unlikely to occur that one has to wonder if there is any validity in the findings.
Very often, an experiment cannot be carried out, but data are collected which show a relationship between two variables - these are correlational studies. The principle in correlational studies is that when one variable changes, another variable changes as well. A positive correlation is when both variables are affected in the same way. As x increases, y increases. For example, the more hours you spend studying, the better you do on exams; or the fewer hours you spend studying, the less well you do on exams. A negative correlation means that as one variable increases, the other decreases. For example, as the number of hours watching television increases, exam scores decrease.
Because no independent variable is manipulated, no cause-and-effect relationship can be determined. For example, a researcher could study the average number of hours that a child watches television and the child's level of aggression. This would be difficult to do as an experiment because it would be unethical. When the data are gathered, the researcher might find that as the number of hours of television viewing increased, so did the level of aggression in the child. This would be a positive correlation. However, it would not be possible to say whether the television viewing caused the aggression, or if it was the aggression which led the child to watch more television. This is called bidirectional ambiguity. It could also be that there is no cause-and-effect relationship at all, but that another variable might be responsible for the behaviour.
Bidirectional ambiguity is seen in correlational research. Since no independent variable is manipulated, it is impossible to know if x causes y, y causes x, if they interact to cause behaviour, or whether it is just coincidental and the results are actually due to a third variable.
ATL: Thinking critically about cause and effect
For each of the following news headlines, do you think that the findings are causal or simply correlational in nature? How do you think that the research would have been done?
1. Cell phones disrupt teen sleep.
2. Using Wash your Hands signs in public bathrooms increases the number of handwashers!
3. Men who are distracted by a beautiful woman are more likely to take financial risks.
4. Study suggests attending religious services sharply cuts risk of death
Checking for understanding
Which of the following is not essential for a study to be a "true" experiment?
It is true that today all experiments should be approved by an ethics board - and if not, they should not be done. However, whether an experiment is ethical or not does not have any effect on whether it is actually an experiment.
What is the key difference between a quasi-experiment and a "true experiment?"
Which of the following is a null hypothesis for a study of the role of aerobic exercise on one's mood?
The final answer is not a null hypothesis, but it is a "two-tailed" hypothesis - that is, it predicts a difference, but it not predicting the nature of the difference. A hypothesis that predicts that the participants will score lower than the other group are stating a "one-tailed" hypothesis.
Which of the following is not an advantage of field experiments?
Which of the following is the best explanation of the difference between a quasi-experiment and a natural experiment?
The final response is actually the opposite of what is true. All natural experiments are quasi experiments in the sense that participants are not randomly allocated to conditions and the IV is not manipulated by the researcher. But natual experiments are usually seen as environmental in nature - e.g. what happened after cell phones was introduced into a village in China? What happened to dietary habits after the hunger famine ended in Holland in World War II?
A researcher carries out a study where the participants were asked to give electric shocks to a student in a study of learning. After the experiment, the researcher asks the participant why he thinks that he was willing to shock the student. He says that he "knew all along that this was fake." This is an example of
The participant may be saying this to make himself look like a "good person." Since he is saying that he knew all along that this was fake and that the person being shocked wasn't being hurt, he is showing that he just "played along" and is not a bad person.
You read in the newspaper that a study of the amount of time people spend on Facebook and one's level of depression showed a strong correlation, but that issues of bidirectional ambiguity could not be resolved. What does this mean?