Week Seven: Small n and Qualitative Designs

When might conducting a true experiment not be feasible?

What is studying a single case?
Click the card to flip 👆
1 / 54
Terms in this set (54)
When might conducting a true experiment not be feasible?

What is studying a single case?
- A researcher might not be able to randomly assign particpants participants to different groups.
-They may not be able to counterbalance conditions in a within-groups experiment.
-It may be unethical or impossible to manipulate particular variables

A researcher may collect data from only one case, such as a single child or a person with an unusual behavioural problem.
How does a quasi experiment differ from a true experiment?

How do true experiments begin their set up?

How is a quasi-experiment different?
Researchers do not have full experimental control in a quasi experiment.

True experiments - start by selecting an independent and dependent variable. They study participants who are exposed to each level of the independent variable.

Quasi-experiment - similar to an experiment except that the researchers do not have full experimental control (e.g they may not be able to randomly assign participants to the independent variable conditions).
Which type of variables do quasi-experiments have?

What is the independent variable called in a quasi-experiment, and why?
Each design has an "independent" variable and a dependent variable.

Quasi-independent variable - because the researchers do not have full experimental control over it.
What are default options?

Why might people favour default optios? Provide 2 reasons.
Default options = opt-in and opt-out procedurues. E.G opting to be an organ doner/ opting out of being one.

People are likely to accept default options when making decisions becuse defaults are effortless.

People may also interpret default options as the socially approved course of action.
Describe how the organ donation example translates into the IV and DV in a quasi- experiment?

Why is this example a quasi-experiment?

Why is this design a nonequivalent control group posttest-only design?
There is a quasi-independent variable (the two default consent options) and a dependent variable (the rate of organ donation).

Researchers did not control which countries had defaults, and people were not randomly assigned to live in the different countires.

Because the participants were not randomly assigned to groups and were only tested once, after exposure to one level of the independent variable or the other.
What is a nonequivalent control group pretest/ posttest design?

Provide an example.
A quasi-experiment that has at least one treatment group and one comparison group, in which participants have not been randomly assigned the two groups, and in which at least one pretest and one posttest are administered.

- 600 patients already elected to undergo plastic surgery.
-250 people who decided not to go through with their surgery.
-Examined before surgery, 3 months, 6 months, a year in.
Why does the surgery example look like an experiment?

However, why is this actually a quasi-experiment?

Why is this an example of a noneqivalent control group pretest/posttest design?
There is an IV (having surgery or not), and a DV (self-esteem, life satisfaction ect)

-The participants were not randomly assigned to the two conditions. (They were self-assigned)
-Lack of random assignment.

The participants were not randomly assigned to groups and were tested both before and after some intervention.
What is an interrupted time-series design?

Provide an example?
A quasi-experimental study in which participants are measured repeatedly on a dependent variable before, during, and after the "interruption" caused by some event.

13 Reasons Why & Suicide: sucide rates in the US were measured - before, during and after the "interruption" caused by the show's release.
What is a nonequivalent control group interrupted time-series design?

Which two designs does the nonequivalent control group interrupted time-series design combine?
A quasi-experiment with two or more groups in which participants have not been randomly assigned to groups; participants are measured repeatedly on a dependent variable before, during, and after the "interruption" caused by some event, and the presence or timing of the interrupting event differs among the groups.

The nonequivalent control group design and the interrupted time-series design.
Using the opioids crisis example, describe how this example fits into the nonequivalent control group interrupted time-series design?

Put simply, what makes this study a quasi-experiment?

Using this example, what are the QIVs and DVs?
Nonequivalent Control Group: The two states were not randomly assigned to having pill mill laws or not.

Interrupted Time Series: researchers did not have experimental control over the year the laws were passed.

QE - the lack of full experimental control.

QIV - pill mill laws (passed or not)
-Time period (before and after the new laws).

DV - overdose rates from prescription opioids.
What do researchers give up when they conducted studies in which they didn't have full experimenter control? How might researchers get closer to making causal claims through a quasi-experiment?Internal validity - the ability to rule out alternative explainations and draw causal conclusions. While not having full control over the IV, they can choose the best possible design and use the pattern of results to rule out some internal validity threats.Which type of design does selection effects impact? When does a selection threat to internal validity apply? Give an example of a possible selection threat in the cosmetic surgery example? How might selection threats be accounted for? How else did the researcher take steps to counteract selection effects?Selection effects are relevant only for independent-groups design, not for repreated measures design. A selection threat to internal validity applies when the participants at one level of the IV are systematically different from those at the other level. 61% of the comparison group reported that they decided against surgery because they couldn't afford it. The pretest/ posttest nature - money explains the increase in life satisfaction, but it does not explain the increase in self-esteem and general attractiveness over time. For some analysis they used matched groups and compaired this data to the full samples. The matched groups version of the data implied that selection effects were not responsible for the differences they observed.What is a wait-list design? What do wait-list designs control for? How could this be applied to the cosmetic surgery example? Why is the wait-list design more similar to a true experiment?An experimental design for studying a therapeutic treatment, in which researchers randomly assign some participants to receive the therapy under investigation immediately, and others to receive it after a time delay. Selection effects. The researcher could have instructed half the participants to receive their surgery right away and place others on a waiting list for surgery later on. They could then measure the patterns of self-esteem, life satisfaction, and general attractiveness in both groups over several months, when only one group would have had surgery. It ensures the same kinds of people are in each group.In certain quasi-experiments, design confounds can be a problem. What are these? Give an example of a possible design confound in the organ donation example? However, before accusing researchers of a design confound, what must you prove?Design confounds - some outside variable accidentally and systematically varies with the levels of the targeted independent variable - this is when the experiment is poorly designed. You might question whether the presumed consent policies co-occur with some other government politicy - public awareness of organ donation. That the alternative explanation was systematic - all seven countries with presumed consent policies had greater awareness of organ donation, while the four opt-in countries did not.When do maturation threats occur? Using the cosmetic surgery example, why might maturation pose a threat? How was maturation threats ruled out?In a pretest and posttest, an observed change could have emerged more or less spontaneously over time. You might ask if improvements in life satisfaction & co was due to surgery or maturation. Comparison group - the results indicated that the comparison group did not improve.When does a history threat occur? Provide possible examples using the opioid Crisis and the 13 Reasons Why example?A history threat occurs when an external, historical event happens for everyone in a study at the same time as the treatment - unclear if outcome is caused by the treatment or an external factor. Opioid Crisis - opioid deaths might have shifted not because of laws passed, but because of changes in living conditions at the same time. 13 Reasons Why - perhaps some other well-publicised event occurred in the country at that time. E.G celebrity suicide.When might history threats be especially relevant to quasi-experiments? What is a selection-history threat? How would you rule out history threats? Using the opioid abuse example.History threats may be especially relevant when a quasi-experiment relies on a historical event to manipulate its key variable. Selection-history threat: the historical event systemtically affects participants only in the treatment group or only in the comparison group, not both. By examining the different states which are diverse - this makes it possible to attribute the opioid decline to laws, rather than different historical events in each individual state.When does regression to the mean occur? When is regression a problem?When an extreme outcome is caused by a combination of random factors that are unlikely to happen in the same combination again - the extreme outcome gets less extreme over time. Regression is a problem when a group is purposefully selected for its unusually low (or high) mean at pretest.In designs with pretests and posttests, when does attrition occur? When does attrition become an internal validity threat? Provide an example. Why was attrition not an issue for the cosmetic surgery example?Attrition occurs when people drop out of a study over time. Systematic drop outs = internal validity threat. E.G people dissapointed with their surgery stopped responding to the study over time. The dropouts were not systematically different from the completers.What is a testing threat? Why can instrumentation also be an internal validity threat? What helps rule out a testing threat to internal validity?A testing threat is a type of order effect in which participants tend to change as a result of having been tested before. E.G improving or declining out of boredom. A measuring instrument could change over repeated uses, and this change would threaten internal validity. A comparison group.Describe the final three internal validity threats that are related to human subjectivity? Describe how you would tackle these threats?Observer bias - the experimenters' expectations influence their interpretation of the results = use a design blind (masked) or double-blind study. Demand characteristics - participants guess what the study is about and change their behaviour in the expected direction = use a double-blind study. Placebo effects - participants improve, but only because they believe they are recieving the effective treatment = include a comparison group.Was observer bias an issue in the cosmetic surgery example? Was the study suspectible to placebo effect? How could you counter this?No, the chance of observer bias was low because the experimenter asked participants to report their own self-esteem and other variables. Possibily, people who have cosmetic surgery are usually aware that they choose to undergo the procedure because it makes you feel better about yourself = increased self-esteem Disguise the fact the study was investigating cosmetic surgery, and instead present a series of questionaires.If quasi-experimental studies can be vulnerable to internal validity threats, why would a researcher use one? What may you not be able to do in quasi-experiments?Quasi-experimental designs - opioid study, cosmetic surgery, 13 reasons - present real-world opportunities for studying interesting phenomena and important events. You may not be able to random assign subjects of interest or manipulate variables - but studying events that occured in real-world settings is still valuable.Which of the validies does quasi-experiments enhance? Why do you frequently not have to ask if quasi-experiments apply to real-world settings? What might you still ask? What do quasi-experiments relinquish control over?The real-world settings of many quasi-experiments can enhance external validity - liklihood that patterns observed will generalise to other circumstances and other individuals. Because real-world settings are where the study took place. You might still ask whether the study's results would generalise to other countries. In general, quasi-experiments capitalise on real-world situations, even as they give up some control over internal validity.What is another reason that researchers might choose a quasi-experimental design? Provide examples.Ethics - many questions of interest to researchers would be unethical to study in a true experiment. It would not be ethical to assign a potentially harmful TV show to young adults, or to assign cosmetic surgery to people who do not want it.What would you do to interrogate the construct validity of a quasi-experiment? Give an example? You also have to ask how successfully the dependent variables were measured in this study?Contrust validity = interrogate how succesfully the study manipulated or measured its variables. 13 reasons why - the researchers took an extra step by using social media data to verify that people were paying close attention to the show in April 17th. How well did the cosmetic surgery researchers measure self-esteem. Did the researchers use measures that were reliable and measured what they intended to measure.How would you assess a quasi-experimental study's statistical validity?You could ask how large the group differences were estimated to be (the effect size) You can also evaluate the confidence interval (precision) of that estimate. (CI)When quasi-exerpiements compare two groups without using random assignment, the groups in the study can look similar to those in other designs. Provide an example. Use an example to describe the similarities between correlational and quasi-experiment designs?Some quasi-experiments seem similar in design to correlational studies. Meeting your spouse online and marital satisfaction - two categories (meeting online or not, and couples were not randomly assigned to either. Seems like a correlational study - 2 measured variables. Seems like a quasi-experiment - two groups of people who were not randomly assigned.What internal validity concerns do quasi-experiments and correlational designs share? How are the two different?Third variables and selection effects. Correlational studies - researchers select a sample, measure two variables, and test the relationship between them. Quasi-experiments - researchers target groups with particular qualities, select a certain time period, or seek out comparison groups provided by state laws. (More intentional).What is a participant variable? Why do participant variables look similar to the nonequivalent control groups of quasi-experiments? What is the intention of studies with participant variables? In contrast, what do quasi-independent variables focus on?Participant variable = categorical variable, such as age, gender, or ethnicity, whose levels are measured rather than manipulated. Both involve measured variables that are categorical. They intend to document similarities and differences due to social identity (gender, social class), development (measured by age), or personality (comparing introverts to extroverts). Quasi-independent variables focus less on individual differences and more on potential interventions such as laws, media, exposure, or education.Should you be concerned with categorising a particular study as either quasi-experimental or correlational?Ultimately, rather than becoming too concerned about categorising a particular study as either quasi-experimental or correlational, or a variable as a quasi-independent or participant variable, focus on applying what you know about threats to internal validity and evaluating causal claims.What do large samples enable? In contrast, describe the sample size in small-N designs?Large samples enable researchers to make more precise statistical estimates. When researchers use small-N design, instead of gathering a little information from a larger sample, they obtain a lot of information from just a few cases.Describe the difference in PARTICIPANTS between large-N designs and small-N designs? Describe the differences in DATA between large-N and Small-N designs?L - PARTICIPANTS are grouped. The data from an individual participant are not of interest in themselves; data from all participants in each group are combined and studied together. S- Each PARTICIPANTS is treated separately. Small-N designs are almost always repeated-measures designs, in which researchers observe how the person or animal responds to several systematically designed conditions. L - DATA is represented as group averages. S - DATA for each individual is presented.Describe the differences in SAMPLES between large-N and Small-N designs? Describe the differences in APPLICATION between large-N and Small-N designs?L - large Samples enable group averags to be estimated more precisely. S - Careful designs enable us to compare each individual during treatment periods and control periods. L - These studies are used for both basic and applied research. S - These designs are often used in therapuetic settings, to confirm that a treatment works for an individual person.Describe the famous case of H.M? Why was this study on memory and neuroscience significant?-H.M suffered from repeated epiletic seizures. -Had part of his brain removed. -Could no longer retain new information/ memories. -Doctors learnt that this part of the brain enables memory formation. Helped neuroscientists and psychologists learn the role of brain regions in memory storage.Why might we question the results of the H.M study? Why might this study effectively advance our knowledge?This research was conducted on very few people. -They used a careful research design. -They used the power of experimental control - comparing his performance to age-matched control participants with no brain damage. -Researchers deliberately designed tasks meant to isolate different elements of memory - they worked hard to present a range of activities to help them distinguis H.Ms perceptional process from memory process.How are small-N designs similar to quasi-experiments? What is a disadvantage of the H.M study? What is another disadvantage? How might ethics prevent creating external validity?Just as quasi-experiments take advantage of natural accidents, laws, or historical events, small-N studies often take advantage of special medical cases. The multiple types of damage to H.Ms brain make it more difficult to narrow down the specific region responsible for each behavioural deficit - internal validity issue? External Validity - participants in small-N studies do not represent the general population very well - E.G H.M had a history of severe epilespy. -Unethical to remove regions of a nonepileptic person's brain to create the necessary comparison.Using the H.M study, what could be done to tackle the issue of external validity? Apart from the H.M example, who else might use small-N studies? When are small-N designs used? What may researchers question in their small-N designs?Triangulate - to compare a case study's results to research using other methods. (parisomious theories and other case studies). In educational, clinical, and work settings, practioners can use small-N designs to learn whether their interventions work. Small-N designs are frequently used in behaviour analysis, a technique in which pracitioners use reinforcement principles to improve a client's behaviour. E.G help a child with autism become less afraid of dogs. Internal validity questions about alternative explanations for the result.What is a stable-baseline design? Provide an example.A small-N design in which a researcher observes behaviour for an extended baseline period before reginning a treatment or other intervention, and continues observing behaviour after the intervention. -Memory technique on real alzheimer's patient. -Practioners spent several weeks recording baseline information (how many words she could remember. -Then she was taught the new techniqueWhy should the results from the alzheimer's example, convince us that the patient's improvement was caused by the new expanded rehearsal technique? What does this provide? What else made this study more viable? What would have happened if the researchers had collected only one baseline measure before the new treatment and a single test afterward?- The baseline was stable - made it unlikely that some spontaneous change happened to occur right when the new therapy began -Provides internal validity - rules out alternative explanations. -Replication - provided further evidence. -The improvement could be explained by any number of factors: maturation, regression, history effect.What is a multiple-baseline design? Provide an example.A small-N design in which researchers stagger their introduction of an intervention across a variety of contexts, times, or situations to rule out alternative explanations. -Three children with autisim who feared dogs. -Several sessions of therapy - where they were praised for moving closer to the dog. -All three showed improvement.In the autism/ dog example, what were multiple baselines represented by? Should the results convince us that this therapy works? Could this example support a causal conclusion?Multiple baselines were represented by three children, starting at different times. 1. Each child's baseline distance to dog showed no improvement until the therapy started. 2. The same pattern was observed in 3 children - rules out alternative explanations, such as an outline event that could have affected ony one child. This multiple-baseline design has the internal validity to support a causal conclusion.What is a reversal design? What does this help to test for and make?A small-N design in which a researcher observes a problem behaviour both before and during treatment, and then discontinues the treatment for a while to see if the problem behaviour returns. Internal validity and allows the researcher to make causal statement - if the treatment is really working, behaviour should improve only when the treatment is applied.When are reversal designs most appropriate? When might reversal designs not work? Why might reverse designs be considered both ethical and unethical?Reversal designs are appropriate mainly for situation in which a treatment may not cause lasting change. An educational intervention - once a student has learned a new skill, it's unlikely this woud reverse or go away. Unethical - to withdraw an effective treatment from a patient/ client. Ethical - because it would be unethical to use a treatment that is not empirically demonstrated to be effective.What is a small-N design? What is a single-N design?Small-N design - a study in which researchers gather information from just a few cases. Single-N design - a study in which researchers gather information from only one animal or one person.How do small-N designs discussed eliminate alternative explanations, thereby enhancing internal validity? What about if there is only one participant in these studies?Researchers designed careful, within-subjects experiments that allow them to draw causal conclusions. Even if there was only one participant, the researchers usually measured behaviours repeatedly, both before and after some intervention or manipulation.Why might we criticise the external validity of a small-N design? What step can researchers take to maximise the external validity of their findings?How can one person represent a population of interest. Even when researchers can demonstrate their manipulation, interention, or procedure replicates, the question of generalizability remains. They can triangulate - combining the results of single-N studies with other studies on animals or other groups.Why might researchers not care if their study generalises to everyone? When might they not care at all? Should we disregard a causal statement if it only applies to one person?They rarely intend to generalise to everyone. E.G if a memory strategy for alzhimer's clients applies to everbody in the world - but they do care about how it generalises in the field of alzhimers. They might not be concerned about generalising at all - E.G focusing on one patient's treatment. In such cases, even if the causal statement applies only to one person, it is still useful.When interrogating a small-N design, you should also evaluate construct validity. When is construct validity easily to achieve? When is construct validity less straightforward? What should they do in this case?-Objective measures such as recording how many numbers someone remembers. When you are recording something that could be percieved as subjective. Use multiple observers and check for interrater reliability.Regarding statistical validity, in single-N designs, researchers do not typically use traditional statistics. However, what should they still do? What normaly provides enough quantitative evidence? In addition, you might think about effect sizes more simply in small-N cases, by asking?However, they still draw conclusions from data, and treat data appropriately. Graphs in many cases provide enough quantitative evidence. By what margin did the client's behavior improve?Quasi-experiments can use independent-groups designs such as what? Quasi-experiments can use within-groups designs such as what? When can a quasi-experiment support a causual cliam. What is this in spite of?Such as nonequivalent control group design and a nonequivalent control group pretest/ posttest design. Such as an interrputed time-series design, or a nonequivalent control group interrupted time-series design. -When a quasi-experiment includes a comparison group and the right pattern of results = causal claim. In spite of participants not being randomly assigned to conditions and the researchers not having complete experimental contol of the independent variable.In quasi-experiments, what do researchers balance?In quasi-experiments, researchers balance confidence in internal validity with other priorities, such as opportunities to study ethically in a real-world situation or to take advantage of a political event.What does small-N studies balance? Does this mean the internal validity is insufficient compared to experiments with larger samples?Small-N studies balance an intense, systematic investigation of one or a few people against the usual approach of studying groups of people. The internal validity of small-N studies can be just as high as that of repeated-measures experiments conducted on larger samples.What are the three small-N designs used in applied settings? Which two validies do Small-N designs establish? How might small-N designs achieve external validity? What might a researcher prioritise over broad generalisability?The stable-baseline design, the multiple-baseline design, and the reversal design. Construct and internal validity. May achieve external validity by replicating the results in other settings. In applied settings, researchers might prioritise the ability to establish a treatment's effectiveness for a single individual over broad generalisability.