Psych 120L Midterm
Terms in this set (94)
What makes good research?
It must be testable and falsifiable.
Basic principles of falsifiability
The theory must be stated so that it can be falsified. The theory can only be falsified, not proven correct. The theory can evolve from adjustments on previous tests/formulations. If the theory/hypotheses are not supported, then the theory in current form might be wrong, may need to be retested, revamped or scrapped.
How to we conduct research?
We use the five steps of the scientific method which are Hypothesize, Operationalize, Measure, Evaluate, Revise or Replicate.
How do we develop a hypothesis?
We can take inspiration from previous theories and research. We can also take inspiration from personal observation such as noticing the behaviors of the people around us.
What is a theory?
A theory is an organized set of principles offered to explain a phenomenon. Theories are tested by testing different hypotheses. Theories can't be tested directly.
What makes a good theory?
A good theory will give you a testable hypothesis or set of hypotheses
How do we operationalize?
Once you know what you want to study, you need to get from the abstract to something you can manipulate or measure.
What are conceptual definitions?
The general, abstract variable is often called the conceptual variable.
What are operational definitions?
The concrete, measurable variable is the operational definition of that variable.
What does operationalizing a concept allow you to do?
It allows you to measure the concept objectively and manipulate the concept in an experiment. Example: number of arm touches, seating distance, number of smiles
What do we need to measure?
You need to measure the variables you include in your study. This is for data collection.
What do your data tell you about your investigation?
statistical analysis and graphical representation of your data for evaluation
When do we need to revise or replicate?
If your predictions have been confirmed you will want to replicate (repeat) your study. If your predictions have been disconfirmed you will want to revise your hypothesis or measurements.
Is science purely objective?
No, science is not purely objective. We interpret our world through our knowledge, beliefs, culture, etc. The scientific method helps us to become more objective, but it cannot entirely remove beliefs from our judgements.
In research it is customary to only use primary research (peer reviewed journal articles) or review articles when you're writing up a research article for study.
Milgram obedience studies (1963)
Normal healthy men were asked to deliver what they thought were extremely painful, potentially harmful shocks to "learner". About 65% of participants complied with authority.
When did the first standards of ethics become published by the APA?
In 1958 following the Hunger study.
How can unethical experimentation be prevented?
Laws, professional practice regulations, internal review boards, informed consent, avoid vulnerable populations, train researchers and practitioners to practice with integrity. Basic principle: do no harm.
Do no harm
Be sensitive to what you are asking your participants to do in the research.
Risks are potential costs pertaining to psychological, physical, social, legal, or economic. You have to consider the "minimal risk". Also consider "who does this effect?" such as participants and their peripheral relationships. The benefits are advancement of science, practical improvements in treatments or interventions, and adequate compensation.
Who is recruited to participate in research?
You need demographic diversity, representativeness of sample, and must have a reason for excluding certain groups. The key question is whether there is coercion- "pressure to take part in or remain in a study".
Under what conditions are people being asked to participate?
Look at their incentives and their willingness to participate.
Informed consent for adults
Participants are informed of exactly what they are getting themselves into. It's a contract between the investigator and participant, usually written and signed. Key components consist of purpose, procedures, risks and benefits, confidentiality of data, instruction that they are free to withdraw consent and discontinue participation at any time, offer to answer questions about these procedures, and contact person for concerns.
Informed assent for children, mental illness, other competency
Assent is obtained when the person is not considered to be legally competent to give informed consent. You need informed consent by the legal guardian as well as an agreement (assent) by the child that he/she understands what he/she will be doing, the risks/benefits involved, and that he/she can withdraw.
Informed consent in designs involving deception
This occurs when the researcher withholds information rather than lie directly in the consent. You should never withhold risk information. And after the session you must debrief the participants about the deception and give the participants the opportunity to withdraw data after a full debriefing.
Do not disclose what participants reveal during the course of the study. This includes not sharing their personal identifiers (ex: name). Exceptions to confidentiality in cases of studies that ask about potential harm to self or others. De-identify data, protect health information, and maintain anonymity.
Providing true choice
Participants need to know they are free to not participate and they can decide this at any time. Make stipulations about destroying data in consent. If withdrawals or strong reactions, might indicate that you need to change your procedures.
Obligation to screen and intervene
Only collect data that the researcher is equipped to handle. Rigorous review of sensitive data. Contingency plans for when disclosures indicate potential harm to self or others. Potential for participants to receive follow-up referrals or information as needed.
Determining appropriate control groups
Problems with untreated control groups when conditions being studied are potentially serious/harmful are "standard of care" as control or "no treatment". Alternatives that help researchers to be more ethical are offer free active treatment to control group after study trial, use wait-list control group, and crucial to inform about potential for placebo in drug studies.
Provide prompt information on the nature of the study, including if deception was used. Must actively try to alleviate any distress/negative consequences caused by the research. Probe for suspicion (if deception study). Provide prompts information on results and conclusions.
What is a hypothesis?
A hypothesis is an explicit, testable prediction about the conditions under which an outcome will occur.
Key features of a hypothesis
They must contain at least two concepts and a statement of the relationship between them. They can be operationalized and tested (theories can only be tested by testing the hypotheses).
What is operationalization?
Operationalization is the process by which we make a theoretical variable one that we can measure. An example: stereotype threat. The fear that we will confirm the stereotypes that others have regarding some salient group of which we are a member. Essentially when a person knows about a negative stereotype the person becomes anxious about fulfilling it and ends up fulfilling the negative stereotype because they are anxious about their performance.
Hypothesis stated with conceptual variables
For women, when the gender stereotype is confirmed, math performance will suffer, while for men, when the gender stereotype is confirmed, math performance will improve.
Hypothesis stated with operationalized definitions
For women, when the test instructions said the test had shown gender differences in the past, this will lead to worse performance on a portion of the math GRE, whereas men who received instructions that said the test has shown gender differences in the past will perform better on a portion of the math GRE.
Independent variable (IV)
What a researcher changes (manipulates) to see if it has an effect on some other variable
Dependent variable (DV)
What a researcher measures to determine the influence of the independent variables
Manipulation vs measurement
IVs are manipulated (so that changes in the DV can be explained only by changes in the IV). There are multiple levels of an IV (ex: warm, cold, hot). There are treatmetn and controls (subjects told about negative effects of smoking or not). The DVs are measured.
What's the point of running an experiment?
The point of running an experiment is to determine whether X caused the differences in Y.
How can we avoid possible confounding in our research?
To solve this, subjects must be randomly assigned to the various levels of X.
What if people in different conditions had different experiences during the study?
If the levels of X differed in any way other than X (ex: run at different times of the day), this becomes a potential third variable or confound. For this reason, attempts need to be made to hold the situations for all condition constant except for differences in X.
A true experiment consists of...
A manipulated IV, random assignment of subjects to the conditions or levels of the IV, and experimental control: reasonable attempts to hold the situation constant other than the manipulated IV. NOTE: if you can't control a possible confounding variable then measure it so you can statistically control for it later
Degree to which a test actually measures what it claims to measure. Did you measure what you intended to measure? Did you manipulate the IV in an effective way? We can check by doing a manipulation check.
An additional measure or question in the study that checks to make sure that your manipulation induced the changes it was assumed to have made. Ex: if you are inducing a mood as an IV then a manipulation check can be a mood questionnaire.
extent to which survey test items or are a representative sample of the behavior measured
extent to which survey items appear on the surface to measure what it intends to measure
example of face validity
Validity Category: If you were going to measure anxiety, does your measure appear to actually measure anxiety? If so, it has THIS validity.
example of content validity
Validity Category: A test of algebra corresponds to the content of the course.
- An IQ test that did not sample the entire range of intelligence (only memory) would have poor content validity.
degree to which there can be reasonable certainty that the IV in an experiment caused the effects obtained by the DV
Can you be sure that X actually caused Y?
The main reason for using experimental methods is to ensure internal validity. If it is a true experiment, then internal validity is usually high.
participants must be randomized to experimental conditions to avoid confounds
Degree to which there can be reasonable confidence that results of a study would be obtained for other people and in other situations
how well people in your experiment represent a random sample of the larger population of interest. Most psych research uses a convenience sample.
this is the extent to which an experiment physically resembles real life situations
this is the extent to which the psychological processes triggered in an experiment are similar to those that occur in everyday life
Have you demonstrated that an effect is not likely due to chance? Statistical significance testing (95% confidence). Obtaining statistical significance can be achieved with a large number of subjects in a study and large experimental effects. Statistical significance does not mean importance, just a degree of certainty that results are not due to chance.
If findings do not support our hypothesis, we revise our hypotheses or operationalizations and go from there.
In scientific research, we require that findings be replicable to ensure that our results were not due to statistical error.
Internal validity is easier to attain in the_________.
External validity is easier to attain in the __________.
Example: goal is simply to describe behavior. We can do a case study, population surveys, epidemiological research, and marking research
Measure 2 or more variables to determine covariance--Are the 2 variables significantly related to each other? We can do a survey, observation in a lab, and field research. These design cannot show causality. Thus, the conclusions are limited to describing associations of variable.
Manipulation of an IV and measure its effects on a DV. The IV is the variable whose values are chosen and set by the experimenter and what gets manipulated in the experimental groups. The DV is the variable whose value is observed and recorded. Participants are randomized to condition. Aim is to show causality.
Equivalent to experimental designs but you cannot assign/randomize participants to a variable that is of importance. Why? Cannot always assign members based on a particular status. Their status might be important so you still measure them.
Random assignment/randomization of participants to conditions
We do this in experimental groups and control groups. The procedure for randomization (ex: flip a coin or random # table). Purpose of random assignment is to decide by some unbiased procedure how the groups will be allocated which provides "equal chance" of biases in each group. This prevents confounds related to people being variable on how they respond to a manipulation.
It's a statement of the expected relationship between your variables.
It's about there being no predicted differences between the mean level of the effects of the experimental conditions on the DV. In an experimental design, the null hypothesis is often stated as no effect of the IV on the DV. If you are running a simple experiment with an IV that has only one experimental group and a control group then the null hypothesis is there is no effect of the IV on the DV. HOWEVER, many research designs are asking more complicated questions than just one experimental manipulation vs. a control.
The alternative hypothesis always needs to be stated such that you are comparing the effects of the experimental conditions on the DV. In research we use the alternative hypothesis in our design. Why do we care about the null then? The null is what your stats test so to understand what our stats are doing we need to also understand the null hypothesis
Using an alternative hypothesis
The alternative hypothesis always needs to be stated such that you are comparing the effects of the experimental conditions on the DV. A good hypothesis will always have some variation of the phrase "as compared to".
Bad hypothesis and Good hypothesis examples
Bad hypothesis: Students who receive positive feedback will score higher on self report measure of the intrinsic motivation. <--no comparison group!
Good hypothesis: Students who receive positive feedback will score higher on self report measure of intrinsic motivation, as compared to students who received no feedback.
Example of alternative hypothesis
The alternative hypothesis always needs to be stated such that you are comparing the effects of the experimental conditions on the DV.
Example: Patients who receive therapy treatment A will show greater improvement in depression scores after treatment compared to those who received therapy treatment B.
Example of null hypothesis
The null hypothesis is about there being no predicted differences between the mean level of the effects of the experimental conditions on the DV.
Example: There will be no difference in improvements in patient depression scores between those patients that received therapy treatment A versus those that received therapy treatment B.
We are saying there is no difference between their effects on depression scores-A is not better than B.
True experimental designs: Between subjects
focuses on the differences between individuals on a given variable/process-->participants are assigned to one treatment condition each
True experimental designs: Within subjects
focuses on differences between multiple measurements of a variable within the person; every person gets all of the same measurements as the other participants
True experimental designs: Mixed
combines measurement at both between and within person levels
uses ANOVA to examine differences between groups
Multi-factorial (multiple IVs)
uses multiple linear regression or ANOVA to examine effects. Examine significance of omnibus F. Examine main effects and interactions. All estimates include effect and error.
How each IV impacts the level of the DV. The effect of IV #1 on a DV. The effect of IV #2 on a DV. The IV has an effect on DV, regardless of the level of your other IV.
IV has effect on DV, regardless of the level of your other IVs
Two IVs work together to determine the level of the DV. The effect of one IV on a DV is different for the different levels of another IV. A significant interaction always qualifies the main effects. That is, the simpler story of main effects are not the most important anymore.
Another variable has an effect on the relationship between the IV and the DV.
Example of stating potential interaction effects: Participants with phone present will report lower relationship quality, with their paired partner, and this will be more pronounced for those engaging in the meaningful conversation. Relationship quality will suffer the most when people are trying to make a meaningful connection and a mobile phone is present as compared to all other conditions.
Type 1 error
You claim there is a relationship between variables that does not actually exist (rejected the null when it is true)
Type 2 error
You failed to claim a relationship between the variables and there actually is a relationship (retained the null when it is false)
What does your p value tell you?
The p value is estimating the probability of make a type 1 error
So, when we say the effect is significant (p<.05), we are actually saying that there is less than a 5% chance that we have made a type 1 error
Less than 5% chance that we claimed a relationship between variable and the relationship does not actually exist.
Inferential stats requires
significance, effect size, and power
It expresses the likelihood that the observed effect would occur by chance alone. The minimum level is p is less than equal to .05. In psychology if <.05 then hypothesis is considered supported. Significance tests do not imply how large the effects are.
It indicates the strength and magnitude of the effect. Correlational (r): strength of association between two variables. Experimental (d): difference between means of experimental and control groups (ex: degree of change in the DV attributable to the IV).
.6-.8 is very large
.3-.5 is moderate
.2-.3 is weak
It is the probability of rejecting the null hypothesis when it needs to be rejected (when it is false). Power is the probability of not making a type 2 error. Same size is one of the major determinants. Power analysis: At outset of study researchers can determine sample size needed to potentially detect effect by assuming a particular effect size.
Drawing conclusions from hypothesis testing
Justified in concluding no relationship between two variables/no differences between experimental and control group when...
1. very small average effect size
2. significance test that doesn't reach acceptable p-value
3. power is acceptable
Tests whether there is a significant difference in the means of two levels of an IV (ex: do men and women differ in their level of neuroticism)
Tells you that at least some of the differences among the means of multiple conditions/levels of a DV are not caused by chance, but instead a product of your IV/predictor variable. F test just tells you there are differences overall and doesn't tell where the differences lie within your conditions. To find the specific differences used planned comparisons or post hoc tests.
Mean of condition
mean of one DV in a particular condition
mean of the effects of one IV collapsed across the other IV
You will have three F values; one for each main effect and one F for the interaction. Each F value will have an associated p value. The p value tells you if that F value is significant.
Sometimes experimenters think that the characteristics/attitudes/predispositions that people bring into the lab may affect how they will react to the experiment manipulation. A moderator variable is a quasi-variable or continuous variable that you measure (you never manipulate it!) and is used to potentially explain how individual differences in participants might impact the main relationship you are test (IV->DV)