Program development review of literature

 

Order Number			45471455714
Type of Paper			Essay
Writer Classification Level			Undergraduate
Style of Writing			MLA/APA/CHICAGO/
Sources and References			5
Number of Pages			4
Paper Instructions Lecture Overview   Lecture Overview • Measurement terms • General considerations: – The aims of measurement – Challenges that can arise • Deciding what to measure • Deciding how to measure • Sampling considerations   Definitions   Definitions of Measurement Terms • Operationalize: to define a concept in a way that it can be measured • Measurement: the use of data to characterize a given phenomenon • Variable: a measured characteristic that varies across participants • Control variable: a variable that influences results by distinguishing between participants   General Considerations We can use measurement to fulfill various aims: • To know whether a program was effective • Improving the design of a program • Testing a new, innovative idea • Changes for future implementations • Continuing, ending, or reenvisioning a program • Deciding whether to continue funding • Fulfilling grant requirements Considerations for Measurement Decisions Measurement decisions should help to answer such questions as: • Did the program reach its goals and objectives? • Was this program more effective than other similar programs? • Who benefited from the program? Did certain participants benefit more/less than others? • What should be improved/changed in the program? • Does the benefit of the program warrant the cost? • Fidelity: Is the program being implemented as designed?   Possible Challenges Challenges that can arise in the measurement process include: • Working with people who do not value research • Working with people who don’t want to be studied • Evaluation can conflict with time demands of directors/staff • Alignment of your goals as an evaluator with agency’s goals • Getting your “foot in the door”/resistance • Obtaining consent to provide data • Logistics: scheduling time and space to collect data   Deciding What to Measure • What is the best indicator of success for your program? • You can define success based on: – What you find interesting – What would be informative to your setting – What would be important to the clients • Outcomes vs. process • Example: job training in an impoverished neighborhood – Control variables: What variable(s) might influence the results by distinguishing between participants?   Deciding How to Measure • Be sure your instruments measure what you want them to measure – Match what you want to know with how you measure it • Common types of measurement tools – Interviews – Observation – Written surveys – Focus groups     Measurement Type: Interviews • In person or by telephone • Open-ended and close-ended questions • Formal interviews—systematic method for obtaining data – Start with a list of topics to be covered and a clear sense of what kind of information is wanted – Structured interviews ask each participant the same questions with the same wording in the same order Measurement Type: Interviews (cont.) • Recorded when possible, transcribed, and then reviewed and coded based on themes • Coding: the practice of taking narrative information and slotting it into a set of categories that capture the essence of their meaning • Interviewer bias: the interviewer might ask questions differently to different people     Measurement Type: Observation • Evaluator develops a protocol for recording specific behaviors during a predetermined observation period – E.g., records the number of questions participants asked, number of participants who engaged in the discussion, etc. • Evaluator takes notes/codes in real time or makes recording and codes later Measurement Type: Observation (cont.) • Unobtrusive observation can have minimal impact on client behaviors – But clients may know they are being observed, which will likely influence their behavior • Observer bias can occur in coding and note taking – Use multiple observers and check interrater reliability     Measurement Type: Written Surveys • Open-ended or close-ended • Surveys, inventories, standardized scales • In-person (individual or group administration) or mailed or online • Be sure to check the reading and writing level of questionnaires • Also check for language that is not suited to your population (e.g., unfamiliar language)   Measurement Type: Focus Groups • Usually five to eight people discussing topic(s) to provide data • Participants can be clients, students, staff, community members • Useful for hearing interplay between ideas • Anticipate power differentials • Be purposeful in selecting group members • How many facilitators? • What are the questions? • How many groups?   All Project Proposals Should Include… • A description of all measures • A description of the participants to whom the measures will be administered • A discussion of strategies and plans for recruitment and retention of participants   Lecture Overview • Study design: general considerations • Description of quantitative design • Types of quantitative design • Challenges in quantitative design • What to include in your proposal     Study Design: General Considerations Study design: How many groups will be studied, at what time points, and what comparisons will be drawn? • The design of your study guides all aspects of data collection and the selection of participants • The main strength of quantitative approaches in evaluation research is that these approaches often allow for the collection of large amounts of data that can be examined using a range of statistical analyses • Many researchers still think of evaluation as being primarily—or exclusively—quantitative     General Challenges in Quantitative Design Challenges in comparing pre- and postscores: • Pretesting can cause change that may not happen in an untested population • Selection: participants in a study may differ from those who do not volunteer for a research study • Maturation: passage of time and growing older may account for observed outcomes • Testing: taking a test repeatedly can change how people score on a test over time   Types of Quantitative Design • Using one group – After only – Before and after – Time-series design • Using comparison group – After only – Before and after – Multiple time-series design Types of Quantitative Design (cont.) • Randomized experiment –Participants randomly assigned to experimental group or control group –Wait-list control: issues? – Importance of attention placebo   Challenges in Using Randomization • Participants may refuse to participate if randomized to a condition they don’t want • A participant may attend very few sessions • Attrition Strategies to deal with these difficulties: • Include participants with low attendance in the analyses—mimics the real world • Try to maintain contact with controls • Offer incentives for remaining in program • Use before data of drop-outs to compare them to those who remained in the study General Critiques of Randomization • “Withholding” treatment from control group • Limited external validity (not generalizable to real world) • Decisions can’t be made until after program has run its full cycle • Inequitable distribution of power between evaluators and program staff • Experiment does not examine the “how” and the “why”: only focuses on outcome     In Your Proposal… • Describe the details of your study design – How many groups will you have? –What comparisons will you make? • Use terms correctly – Control vs. comparison group – Clients vs. participants
	Distinguished (100%)	Proficient (85%)		Basic (70%)	Below Expectations (50%)	Non-Performance (0%)
Thesis Statement	Raises the strongest   objection to the thesis presented in the assignment. The objection is strongly grounded in research and logical reasoning.	Raises a plausible objection to the thesis presented in the assignment. The objection is mostly grounded in research and logical reasoning.		Raises an objection to the thesis presented in the assignment. The objection is somewhat grounded in research and logical reasoning.	Attempts to raise an objection to the thesis presented in the assignment. The objection is minimally grounded in research and logical reasoning.	The objection to the thesis is either nonexistent or lacks the components described in the assignment instructions.
Counter Argument	Provides a strong, thorough rebuttal to the objection. The rebuttal effectively demonstrates that the thesis can withstand the objection and applies the principles of charity and accuracy.	Provides a rebuttal to the objection. The rebuttal mostly demonstrates that the thesis can withstand the objection and mostly applies the principles of charity and accuracy.		Provides a limited rebuttal to the objection. The rebuttal somewhat demonstrates that the thesis can withstand the objection and somewhat applies the principles of charity and accuracy.	Attempts to provide a rebuttal to the objection; however, the rebuttal minimally demonstrates that the thesis can withstand the objection and does not apply the principles of charity and accuracy.	The rebuttal is either nonexistent or lacks the components described in the assignment instructions.
Conclusion	Provides clear and concise closing remarks that comprehensively summarize the essay. The remarks consider the broader controversy and/or further research that could offer additional insight into the moral solution of the business problem.	Provides closing remarks that summarize the essay. The remarks mostly consider the broader controversy and/or further research that could offer additional insight into the moral solution of the business problem. The closing remarks are somewhat unclear.		Provides closing remarks that minimally summarizes the essay. The remarks minimally consider the broader controversy and/or further research that could offer additional insight into the moral solution of the business problems. The closing remarks are unclear and/or vague.	Attempts to provide closing remarks that summarize the essay; however, the remarks do not consider the broader controversy and/or further research that could offer additional insight into the moral solution of the business problem. The closing remarks are unclear and vague.	The closing remarks are either nonexistent or lack the components described in the assignment instructions.
Written Communication: Context of and Purpose for Writing	Demonstrates methodical application of organization and presentation of content. The purpose of the writing is evident and easy to understand. Summaries, quotes, and/or paraphrases fit naturally into the sentences and paragraphs. Paper flows smoothly.	Demonstrates sufficient application of organization and presentation of content. The purpose of the writing is, for the most part, clear and easy to understand. There are some problems with the blending of summaries, paraphrases, and quotes. Paper flows somewhat smoothly.		Demonstrates a limited understanding of organization and presentation of content in written work. The purpose of the writing is somewhat evident but may not be integrated throughout the assignment. There are many problems with the blending of summaries, paraphrases, and quotes. Paper does not flow smoothly in all sections.	Organization and presentation of content are extremely limited. The purpose of the writing is unclear. There is little or no blending of summaries, paraphrases, and quotes. Paper does not flow smoothly when read.	The assignment is either nonexistent or lacks the components described in the instructions.
Written Communication: Control of Syntax and Mechanics	– Displays meticulous comprehension and organization of syntax and mechanics, such as spelling and grammar. Written work contains no errors and is very easy to understand.	Displays comprehension and organization of syntax and mechanics, such as spelling and grammar. Written work contains only a few minor errors and is mostly easy to understand		Displays basic comprehension of syntax and mechanics, such as spelling and grammar. Written work contains a few errors which may slightly distract the reader.	Fails to display basic comprehension of syntax or mechanics, such as spelling and grammar. Written work contains major errors which distract the reader.	The assignment is either nonexistent or lacks the components described in the instructions.
Written Communication: Required Formatting	Accurately uses required formatting consistently throughout the paper, title page, and reference page.	Exhibits required formatting throughout the paper. However, layout contains a few minor errors.		Exhibits limited knowledge of required formatting throughout the paper. However, layout does not meet all requirements.	Fails to exhibit basic knowledge of required formatting. There are frequent errors, making the layout difficult to distinguish as required style.	The assignment is either nonexistent or lacks the components described in the instructions.
Written Communication: Word Requirement	The length of the paper is equivalent to the required number of words.	The length of the paper is nearly equivalent to the required number of words.		The length of the paper is equivalent to at least three quarters of the required number of words.	The length of the paper is equivalent to at least one half of the required number of words.	The assignment is either nonexistent or lacks the components described in the instructions.
Written Communication: Resource Requirement	Uses more than the required number of scholarly sources, providing compelling evidence to support ideas. All sources on the reference page are used and cited correctly within the body of the assignment.	Uses the required number of scholarly sources to support ideas. All sources on the reference page are used and cited correctly within the body of the assignment.		Uses less than the required number of sources to support ideas. Some sources may not be scholarly. Most sources on the reference page are used within the body of the assignment. Citations may not be formatted correctly.	Uses an inadequate number of sources that provide little or no support for ideas. Sources used may not be scholarly. Most sources on the reference page are not used within the body of the assignment. Citations are not formatted correctly.	The assignment is either nonexistent or lacks the components described in the instructions.