Get ahead with a $300 test prep scholarship
| Enter to win by Tuesday 9/24
Research Ch. 12
Terms in this set (57)
Explain Program Planning
Allows the counselor to set the design of their programs; provides counselors with the opportunity to identify the purposes, goals, objectives, and activities of the program
(ensures that services are carefully thought out before delivery)
Explain Program Implementation
pertains to managing the details of program delivery; insuring that the materials needed to deliver a program are available for use at the time they are needed.
Explain Program Evaluation
pertains to making value judgments about the totality of counseling program, including their processes and outcomes
Counselors are committed to providing what? Specifically counselors must attend to the tasks of?
-effective, high quality counseling programs and services to the clientele whom they serve
-program planning, program implementation, and program evaluation
Context Evaluations are?
assessing the needs of the clientele who will be served by a counseling program, determining if programs that can address these needs are already in place, and deciding if there are sufficient institutional resources to offer the program.
the systematic collection of evidence that will allow for the appraisal of the quality of a counseling program and assist in making decisions that will change counseling service delivery for the better.
Explain the 6 steps of the Six Step Model of Evaluation
i. Define the mission of the program
ii. Translate mission into goals
iii. Identify evaluation questions and data collection strategies
iv. Collect, compile, analyze, and interpret data
v. Using results, evaluate the value and worth of program elements and as a whole
vi. Using decisions from step 5, improve program design and delivery or discontinue the program
Primary purpose of program evaluation is?
determining merit and worth because it emphasizes the valuing component of evaluation
Three differences between evaluation and research?
Purpose, control and generalizability of results
In regards to purpose, what is the difference between evaluation and research?
evaluations- basis for making judgements about the value of program
research-to identify new knowledge for a discipline or field of study
In regards to control, what is the difference between evaluation and research?
evaluations- control dispereds among audience of stakeholders
research- rests solely with primary investigator
In regards to generalizability of results, what is the difference between evaluation and research?
evalutaions- program evaluators are only interested in describing effects of a particular person
research- important to demonstrate the results of an experiment can be extended (generalized) to a larger audience
What is Evaluation Model?
alternative strategies that counselors can use to evaluate their programs.
How can evaluation models be differentiated?
(a) primary purpose of the evaluation,
(b) criteria for making value judgments,
(c) consensual assumptions of the model,
(d) anticipated outcomes of the evaluation, and
(e) methods used to conduct the evaluation.
What is quantitative, qualitative, or behavioral data that is used to make judgments or decisions about the value and outcomes of counseling programs?
What is the environmental assessment of available resources in the work environment, including personnel, space and time considerations?
Process evaluation (also referred to as formative evaluation) refers to?
finding answers to questions that pertain to the implementation of a program and opportunities to make revisions to the implementation plan; can also be used to monitor the progress and outcomes of planned activities and interventions.
Product evaluation (also referred to as summative or outcome evaluation) is used to?
identify the extent to which the goals and objectives of a program have been met and to make decisions about program continuation, improvement, or discontinuation.
Product evaluations seek to answer what two questions?
1. Are there changes in the target audience that can be related to the goals and objectives of the program?
2. (More importantly) Can the observed changes be attributed to the program?
Refers to anyone who is involved with, invested in, or may benefit from a counseling program?
The regular, on-going, comprehensive method by which a counselor conducts evaluations of all counseling programs and student outcomes
Utilitarian models depend on ?
identifiable objectives, operational definition of key concepts, and numeric data
3 types of Utiltarian models?
Primary purpose of the Objectives-Oriented Model in CPE?
Purpose:to determine the extent to which program objectives have been achieved; goal is to identify discrepancies between the stated objectives and observed outcomes
Goal: identify discrepancies between the stated objectives and observed outcomes
Example of Objectives Oriented Model?
Provus discrepancy model (purpose is to decide whether to improve, maintain or discontinue a program, or some element of it)
Management-oriented CPE assists with? contains?
- assists with making rational decisions about programs and to evaluate all of the phases and elements of counseling programs, including program development, program planning, program delivery and management, and program outcomes
-contains a comprehensive focus, sensitivity to the informational needs of decision makers, and a systematic approach to decision making.
Example of Management-oriented model?
CIPP (context, input, process and product evaluations)
-provides managers with a framework to use to consider each CIPP element
Explain Process Evaluations
used to provide answers to questions concerning the implementation of the program.
(Is it going well? Are there threats to the success of the program? Does the implementation plan need to be revised in order to guarantee success?)
Explain Consumer-oriented CPE model?
provide information about programs to potential consumers; to provide consumers with a complete picture of all of the effects of a program
Example of the consumer-oriented model?
Scriven's goal-free model (to provide consumes with complete and unbiased view of product or program)
What model assumes that the results will be most beneficial for those individuals who are responsible for implementing the program; rely primarily on judgments of experts?
2 types of Intuitionist/pluralist models ?
Expertise-oriented and participant oriented
Purpose of Expertise-oriented model?
to provide professional judgments about the quality of a program as compared to a set of agreed upon criteria. (Typically, decisions about program quality are made by experts who possess expertise in the discipline and experience in making such judgments)
Judicial and adversary model is based on the notion that?
that judgments about program quality should be based on human judgment and testimony.
4 steps to the Judicial and adversary model?
a. issue generation
b. issue selection
c. preparation of arguments
d. clarification forum
Explain the Accrediation model?
programs that are interested in establishing and promoting these professional standards join together to form accrediting bodies, which in turn formalize the training standards for the profession
(oldest of the evaluation models)
What model is focused on the first-hand observations of individuals who had experience with the goals, activities and direction of the program?
Example of Participant-oriented evaluation model?
Stake's responsive evaluation model
Under Stake's responsive evaluation model, what 3 steps are taken to conduct a responsive evaluation?
a. initial planning and focusing
b. conducting observations
c. organizing and reporting results
(focus may remain fluid depending on findings)
What is a specific behavior that an individual (program participant) exhibits that may be observed and counted?
(I.e.: if wanting to evaluate effectivness of program dsinged to reduce alcohol use among adolescents, arrest records can be a behavioral indicator)
3 approaches that methods can be placed into to collect evaluation information?
behavioral, group or individual approaches
Advantage of usuing a behavioral indicator? disadvantage?
advantage: easy to use because database of interested is always maintained. Also, data obtained is usually highly reliable and valid
disadvantage: may be of limited value when evaluation programs that focus on complex issues (ie: evaluate program designed to promote cultural inclusiveness)
Most widely used group approach for obtaining program evaluation data? it is also the easiest way to obtain date from large groups of stakeholdres
Development of a survey involves what 6 steps?
1. identification of specific topics and issues
2. generation and pilot testing of survey items
3. determination of the sample
4. administration of the survey
5. data analysis
6. interpretation and dissemination of results
3 types of group approaches for collecting evaluation information?
Opinion survey, focus group interviews and key informant evaluation
What are an in-depth interviews with people who know what is going on in the community?
Key informant interview
Using the key informant interview method involves what 5 steps?
1. determine what information is needed
2. develop questions that will allow for the collection of needed data
3. identify and select key informants
4. conduct the interviews
5. compile, organize and interpret the data
What is it called when an individual is evaluated on their ability to display a defined behavior in a particular context?
What is a Portfolio assessment?
assessment of the individual's compilation of a variety of types of evidence that demonstrates ability to perform certain instructional, learning, or behavioral tasks
Under systematic program evaluation, explain the role as a "user"
Counslors must identify critical concepts or issues they want to measure then select n develop measures that provide useful info. Once the "user" decisions have been made, counselor adapts role or "supplier" or "gatherer"
Under systematic program evaluation, explain the role as a "supplier" or "gatherer"
Counselor acts to collect info from consumers and stakeholders, code it, and then store it for later use. Then assume role of "producer"
Under systematic program evaluation, explain the role as a "producer"
work with stored info to produce results - reorganize, analyse and integrate data for the purpose of reporting outcomes then may be the "user" again and use the results to make decisions about value and worth of program or to modify it.
The Accountability Bridge Model for Counselors does what?
organzies counseling evaluation into 2 reoccuring cycles (program and context evaluation) that represent the continual refinement of service based on outcomes, stakeholder feedback and the needs of the population served.
Counseling program evaluation focuses on?
Counseling context evaluation considers?
(both are cycles of the ABM-C model)
Program: delivery and outcomes of services provided
Context: considers the impact of counseling services on users and stakeholdesr
Components of Context Program evaluation cycle include?
1. program planning
2. program implementation
3. program monitoring
4. outcomes assessment
Components of Context Evaluation Cycle include?
1. feedback from stakeholders
2. needs assessment
3. strategic planning
4. service objectives
Roadblocks to evaluation include:
-Buy-in from participants
-Staff has fear of negative results and possible consequences
-Guidance services are often non-systematic, not lending themselves to evaluation
-Some counselors believe that their work cannot be quantified and evaluated
-Time is always insufficient, so evaluation is given low priority