Upgrade to remove ads
Only $35.99/year

Week 9: Systematic Reviews and Meta-Analysis

Terms in this set (18)

What is a systematic review?

Is a systematic review qualitative or quantitative?
A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review.

Systematic review = qualitative analysis of literature.
Simply put, what is meta analysis?

Is meta-analysis a qualitative or quantitative technique?

What is its process?
Meta-analysis = where researchers combine the findings from multiple studies to draw an overall conclusion.

A quantitative technique for synthesising the results of multiple studies of a phenomenon into a single result.

Process = combining the effect sizes estimates from each study into a single estimate for the combined effect size or into a distribution of effect sizes.
What is meta-analysis ideally suited for?

Give an example of meta-analysis that a researcher could conduct?

What are some limitations of meta-analysis? Provide 2 answers.
It is ideally suited for summarising a body of literature in terms of its impact, limitations, and implications.

Meta-analysis conducted of several studies on the association between self-efficacy and achievement, integrating the findings into an overall correlation.

-There is no minimum number of studies nor participants required.

-Information of potential interest may be missing from the original research reports upon which the procedure must rely.
Describe the process of a systematic review?
- Starts with identifying records systematically in the literature.

- Filters which records to keep and which to discard.

- Pulls them together for analysis.

-Reports the findings out.
What is the difference between a systematic review and and meta-analysis?
A systematic review attempts to gather all available empirical research by using clearly defined, systematic methods to obtain answers to a specific question.

A meta-analysis is a statistical process of analysing and combining results from several similar studies.
What is the difference between a systematic review and a systematic literature review?
Literature reviews usually answer broad and descriptive research questions.

Systematic reviews are more comprehensive and precise because they seek to answer specific scientific questions of high importance. (More rigorous than a literature review).
What is a systematic literature review?

What type of research project is a systematic review?

What are the 6 crucial steps for a systematic literature review?
SLR = identifies, selects and critically appraises research in order to answer a clearly formulated question.

It is a type of secondary research - its own research project.

1. Determine your question.
2. Determine your procedures.
3. Determine your sample.
4. Determine your data.
5. Analyse your data.
6. Reporting and Discussing Findings.
Step 1 = Determine your question. What is the first thing you should do here?

Why should you do this?

When might duplication be valid?

Why must you strike the balance in specificity in this research?
Look for registered protocols/ existing meta-analyses or systematic reviews on your topic.

Because you do not want to duplicate work that has already been done.

If a systematic review or meta-analysis is out of date = course to duplicate.

Must be appropriately specific, yet if it is too specific there might not be enough literature on the topic.
Step 2 = Determining your Procedure. What must this include?

What else do you need to determine?

What should you be weary of?

Who is it recommended that you should speak to?
Determining your procedure should include your inclusion and exclusion criteria - by which you filter the research studies that you find.

You need to determine your search approach as well as your search databases (e.G Pubmed, Psychinfo).

Be weary of unpublished or grey literature.

Recommendation - talk to a librarian about your search strategy.
Step 3 = Determine your Sample.

Why might published literature be similar to the population? (metaphor)

What must you do during this process?

Ideally, what should you have? And subsequently, what will you calculate?

Give a scenario example where there might be a difference in coders opinions. What must be done here?
Imagine all the published literature in the world is the population of studies = you must sample from them strategically.

- You must use a systematic and rigorous approach which is specified in advance.
- You must filter your data screen in and say why, screen out and say why.

Ideally, you will have at least two coders = each item will be double coded.

You must calculate inter-rater reliability between the coders.

One coder screens a paper in, another screens a paper out = would need to be resolved (perhaps bringing in a third person).

You would need to document this disagreement and come up with a final resolution.
Step 4 = Determine your Data.

What are you doing here?

What are possible attributes may you be looking at?
You must extract your data = what are the attributes you are observing (these should have been decided in advance)

-Sample characteristics.
-How specific constructs are measured.
-Quality of assessment tools.
-Risk of bias assessments - lots of tools available for the specific research you are doing.
Step 5 = Analyse your Data.

What does this stage depend on?
This stage differs depending on if you are doing systematic review or meta-analysis.
Step 5 = Analyse your Data.

For a systematic review, what might you want to put together?

What done this in turn create?

How can you learn how to formulate this?
For a systematic review, you are going to want to put a table together that has the key components that you extracted your data from.

Table = comprehensive set of data.

Look in journals where you anticipate publishing your findings in advance, and replicate the table format.
How does meta-analysis go beyond the systematic review table?

Describe how meta-analysis works in terms of weighting?

Why do bigger studies get heavier "weight"?

What will you continue to do before ultimately concluding?
Meta-analysis may also have a summary table.

The next step will be quantitatively combining and analysing your data.

The results are weighted by the size of each studies - bigger studies get a heavier "weight" than smaller studies".

Because they have more accurate effect size, all things considered equal.

You will continue to weigh them as you go through each study - based on sample size. Then eventually you conclude which way the weight of evidence flows.
In the presentation of meta-analysis studies from a systematic review, what is typically shown on the Y and X axes?
X axes = standardised mean differences.
Y axes = the different origins of where those studies came from.
Step 6 = Reporting and Discussing Findings.

What should you comment on during this stage?
-Comment on the overall quality of the literature.

-Comment on what you have learned.

-Link back to your original research question(s).
How do we judge the quality of a systematic review?
-What is the research question (using PICO).
-What is the scope of the literature that was reviewed.
-Have they included literature which opposes their general view to account for bias.
-Examine the inclusion and exclusion criteria.
-How many coders or reviewers were there - are there at least 2, and what percentage of articles did they review.
What is gray literature? What does this include?
Gray literature = studies that haven't been published or studies that have been published but in unacademic formats, but academic research has been conducted.

Gray literature = dissertations, presentations, manual scripts, abstracts from academic resources.
Other sets by this creator