Evaluation Design and Methods
Number of unique resources found: 78
Did you find what you were looking for? If not, try our full text search or let us know by completing our survey form.
Description: This resource website focuses on qualitative methodologies, particularly action research.
– Evaluation –
A Ranking Question for a Needs Assessment 
Author: Kiernan, Nancy (3 more by this author)
Description: This tip-sheet examines the use of ranking when conducting a needs assessment.
Action Research Electronic Reader
Author: Hughes, Ian ; Bob Dick
Description: This website provides an introduction to action research and discusses specific methods, e.g., observing, diary, focus groups, etc. There are also links to several related articles and resources focusing on action research.
Adapting Evaluation Measures for Hard to Reach Audiences
Author: Earthman, Erik; Marczak, Mary S.; Peterson, Donna J.; Richmond, Lucinda S.
Description: We are frequently unable to obtain evaluation information from participants with limited English or low literacy skills young children, and people from non-majority cultures. Together these groups represent the "hard to reach" audiences of evaluation. This bulletin provides practical guidelines for program evaluators working with "hard to reach" audiences.
Alternative Methods for Collecting Evaluation Data
Description: Collection of documents discussing the use of non-traditional methods for the collection of evaluation data. Topics covered include: focus groups, cost analysis, portfolio assessment, qualitative interviews, and existing records.
An Educator's Guide to Evaluating The Use of Technology in Schools and Classrooms
Author: Quiñones, Sherri ; Rita Kirshstein
Description: This guide is designed to help you through the technology program evaluation process and to make it as painless as possible. Primarily geared to educators or administrators with little or no research experience, it is intended for use at the district or school level.
An Overview: Designing Initiative Evaluation
Author: Parsons, Beverly
Description: This resource is designed for external evaluators who conduct initiative evaluations for the W.K. Kellogg Foundation (WKKF). It presents a systems-oriented framework and four general designs for initiative and cluster evaluation.
Description: This guide to technology program evaluation has been developed into a full suite of resources, tools, and professional development activities designed to assist program and project leaders in collecting and analyzing data they can use in making decisions about the implementation of their projects. The framework's elements can be easily adapted for the evaluation of other types of grants and school improvement initiatives.
CYFERnet Exemplary Program Evaluations
Author: Peterson, Donna (5 more by this author); Mancini, Jay A.; Marek, Lydia I.; Betts, Sherry C.
Description: Extension's capacity to evaluate community-based programs has increased dramatically. In 2001, the increased capacity was documented and celebrated. CYFERnet-Evaluation collaborators at the University of Arizona and Virginia Tech solicited nominations of State Strengthening and New Communities Projects that conducted exemplary program evaluations.
Developing a Logic Model: Teaching and Training Guide
Author: Taylor-Powell, Ellen (6 more by this author); Ellen Henert
Description: A new resource for those teaching or mentoring community groups, Extension colleagues or others in creating and using logic models. The Guide includes ice breakers, FAQs and 17 hands-on activities with printable handouts for basic understanding and development of logic models. There is also a separate power point file to complement the guide.
Developing Process Evaluation Questions
Description: This brief provides criteria and examples to help in the development of process evaluation questions.
Description: This brief discusses what logic models are, types of logic models, how to read them, and steps to develop them.
Do's and Don'ts of Community Surveys
Author: Guy, Stan (1 more by this author)
Description: A brief fact sheet on basic tips for conducting surveys in the community. It includes basic and general guidlines involved in conducting community surveys.
Enhancing Program Performance with Logic Models
Description: This website links to an on-line course about the use of logic models in program evaluation.
Evaluating HIV Education Programs
Author: Popham, W. (3 more by this author)
Description: Five key guidelines are described to provide a step-by-step model for the evaluation of an HIV education program.
Evaluation and Intentionality: You Can't Have One Without the Other
Description: Intentional programming and program evaluation go hand in hand. Without intentionally identifying and programming towards targeted outcomes it becomes impossible to develop a pertinent evaluation. This session looks specifically at issues related to both topics including the development of logic models, program fidelity, the black box, and the design and implementation of effective evaluations. Participants will gain an understanding of the relationship between intentional programming and evaluation and have opportunities to apply this knowledge to their own programs.
Description: This publication focuses on promising practices and strategies for evaluating child services; from the Harvard Family Research Project ( available online and in print).
Evaluation Made Easy: Examples of Practical Evaluations - Using Timelines
Author: Jakes, Susan (21 more by this author); Garcia, Aleta; Peterson, Donna J.; Cassidy, Lisa
Description: This bulletin includes a brief introduction to a key indicator, a case example of a community-based program making a difference in the indicator highlighted, sample program goals and objectives, description of program intervention, a sample evaluation plan, key evaluation results, reporting hints and other resources.
Evaluation of HIV Prevention Programs using Qualitative Methods
Description: The three most commonly used qualitative methods are presented. These techniques can provide valuable information about program implementation, participant satisfaction, evidence of program effectiveness, and the identification of areas needing improvement. Qualitative measures can play an important role in an overall program evaluation effort.
Evaluation of Parenting Education Programs: A Parenting Evaluation Decision Framework
Author: DeBord, Karen (61 more by this author); Robert Fetsch; Wally Goddard; Wendy Stivers
Description: As part of the National Network for Family Resiliency, a parenting education special interest group (SIG) was organized in 1996 to examine evaluation models for Extension Educators across the nation. The result is the Parenting Evaluation Decision Framework. The framework includes: Needs and asset assessment; Issue identification; Goal writing; Objective writing; Program intervention; Assessing outcomes; Reporting results; and Redesigning program based on evaluation results.
Author: Grudens-Schuck, Nancy; Larson, Kathlene; Lundy-Allen, Beverlyn
Description: This document briefly describes some fundamental aspects of focus groups, and distinguishes them from other familiar research tools.
From Good Intentions to Effective Interventions: A Primer on Evidence-Based Prevention Programs
Author: Small, Stephen (3 more by this author)
Description: This presentation will provide an overview of evidence-based programs and practices for strengthening families and preventing youth problem behaviors. Participants will gain a deeper understanding of what it means for a program to be "evidence-based" and "cost-effective," as well as insights into how the accumulated evidence on effective programs can be used to improve existing programs and practices more generally. The presentation will conclude with a discussion of the advantages and drawbacks of using evidence-based programs and the role that Extension might play in their dissemination and use.
Author: Chinman, Matthew (1 more by this author); Abraham Wandersman; Pamela Imm
Description: This guide was created by a team of substance
abuse prevention researchers dedicated to helping community organizations reach positive outcomes through a process of
answering ten questions that contain all the elements needed for successful planning, implementation, and evaluation (PIE).
This manual represents a collaborative effort to synthesize evidence-based knowledge and translate it into evidence-based practice.
Author: Chinman, Matthew (1 more by this author); Abraham Wandersman; Pamela Imm
Description: This document contains the tools for Getting To Outcomes, including examples for needs assessments, focus groups, process evaluations, data collection and analysis, satisfaction surveys, etc.
Description: This brief provides a checklist and examples to help assess program goals and objectives.
How Can I Assess the Quality of My Program? Tools for Out-of-School Time Program Practitioners
Author: Kahn, Jordan (1 more by this author); Jacinta Bronte-Tinkew, Ph.D.; Christina Theokas, Ph.D.
Description: This brief identifies some of the advantages of using program quality assessment tools that have been developed for self-assessment and program improvement, and suggests a number of quality assessment tools currently available to assess program quality.
Integrating Logic Models and Work Plans
Description: This brief will define logic models and work plans and the interrelationship between the two; suggest how programs can visually link these two; and explain why linking them is beneficial.
Issues in Evaluating Parenting Curricula
Author: Powell, Douglas
Description: This piece addresses some of the methodological flaws are prevalent and may contribute to the finding of limited program efficacy in parenting education curricula. The author shares lessons learned in terms of (1) incorporating evaluation into all phases of program development and implementation, including program design decisions; (2) employing a theoretical or conceptual model of the anticipated mechanisms of change in parenting programs; and (3) systematically examining for differential program effects in analysis' of program outcomes.
Description: This brief illustrates the basics of logic model creation and implementation.
Making a Difference: Needs Assessment for Coalitions: CDFS-9
Author: Archer, Thomas; McCaslin, N. L.; Cripe, Robert
Description: A fact sheet that discusses needs assessment, including how to plan and design a needs assessment and how to report and use the information gained.
On-Line Evaluation Resource Library
Description: This resource was developed for professionals seeking to design, conduct, document, or review project evaluations. It makes available evaluation plans, instruments, and reports for National Science Foundation projects that can be used as examples.
Online Evaluation Resource Library
Description: The Online Evaluation Resource Library (OERL) is meant for professionals seeking to design, conduct, document, or review project evaluations. OERL is funded by the National Science Foundation.
Outcome Evaluation: A Guide for Out-of-School Time Practitioners
Author: Allen, Tiffany; Jacinta Bronte-Tinkew
Description: This resource provides a basic review of outcome evaluations, discusses why they are important and when they are useful, and discusses ways they can be implemented in out-of-school time programs. It also highlights the experience of an out-of-school time program that has conducted an outcome evaluation and provides a list of helpful evaluation resources.
Description: This guide provides an overview of surveys in social science research. It discusses the many kinds of surveys and provides examples. It also includes information on the process of designing and conducting surveys, and analyzing and reporting on survey results.
Parental use of Child Care: A Guide for Constructing Parent Surveys
Author: Emlen, Arthur; Roberta Weber
Description: This is a guide for constructing surveys on factors driving parental use of supplemental child care. It includes discussions of constructs and variables related to household, parent, work, and child characteristics, child care arrangements, and parent choice.
Practical Guide to Monitoring and Evaluation of Rural Development Projects
Description: This resource provides information on the development, installation and use of effective project-level monitoring and evaluation systems as tools for impact-oriented management of rural development projects.
Probing Questions in Interviews
Description: This short brief offers several practical suggestions for using probing questions when conducting qualitative research through interviews.
Author: Boeree, C. George
Description: This resource was created for Students/Faculty in introductory qualitative methods courses. It provides a broad look at qualitative methods including phenomenology, structural analysis,observation and interviewing.
Quasi-Experimental Evaluations
Author: Moore, Kristin Anderson (7 more by this author)
Description: This brief describes the varied types of quasi-experimental evaluations, what can be learned from such studies, and when it is appropriate to conduct them.
Questionnaire Design: Asking Questions with a Purpose
Author: Taylor-Powell, Ellen (6 more by this author)
Description: This resource goes step-by-step through the process of creating an effective questionnaire.
Random Assignment Evaluation Studies
Author: Moore, Kristin Anderson (7 more by this author); Allison Metz, Ph.D.
Description: Random assignment experimental studies involve using a "lottery" system to randomly assign participants to either a treatment group that receives program services or a control group that does not receive these services, and then comparing outcomes for the two groups. Only rigorous experimental studies can definitively establish that a program causes changes in outcomes for children and youth. This brief explains why random assignment studies are important, how to conduct them, and addresses reservations that program providers might have.
Reflective Assessments: A Tool for Learning
Author: Edwards, Sharon; Cutler, Ira
Description: This article describes the power of "evaluation", a method organizations can use to measure program impact. Articles in this issue describe various options—reflective assessment, risk analysis, standards of excellence, balanced scorecard, matrices and portfolios, and an outcome-based information system.
Research Methods Knowledge Base
Author: Trochim, William (1 more by this author)
Description: This resource provides a comprehensive introduction to social science research methods. It covers the entire research process including: formulating research questions, sampling, measurement, research design, data analysis, and writing a research paper.
Sample Size for Community Needs and Resource Assessment Surveys
Description: A fact sheet that discusses issues concerning sample size for community surveys.
Sampling and Surveying Handbook
Description: This resource provides a broadly applicable introduction to designing surveys or sampling for surveys.
Author: Galloway, Alison
Description: This resource provides a hands-on approach to learn questionnaire design, sampling methods, etc.
The 2002 User Friendly Handbook for Project Evaluation
Author: Frechtling, Joy (2 more by this author)
Description: This publication serves as a guide for evaluating the National Science Foundation's educational programs. It discusses both quantitative and qualitative evaluation methods.
The Use of Portfolio Assessment in Evaluation
Author: Sewell, Meg (2 more by this author); Horn, Melanie; Marczak, Mary
Description: Discusses the use of portfolio assessment for evaluation. Topics include how and when to use portfolios for evaluation, and the advantages and disadvantages of doing so. Also includes an annotated bibliography on the subject.
The Use of Qualitative Interviews in Evaluation
Author: Sewell, Meg (2 more by this author)
Description: Discusses the use of qualitative interviewing for evaluation. Topics include types of qualitative interviewing and how and when to use them for evaluation. An annotated bilbliography on the subject is also included.
User's Guide to Evaluation for National Service Programs
Description: This evaluation guide is widely applicable to beginners conducting internal evaluations of a service program. The guide is designed to assist programs by addressing four key priority areas: education, public safety, human needs, and environment.
Using an Evaluation for a Needs Assessment
Author: Kiernan, Nancy (3 more by this author)
Description: This tip-sheet discusses the challenges and benefits of using an evaluation for a needs assessment.
Using Cost Analysis in Evaluation
Author: Sewell, Meg (2 more by this author); Marczak, Mary
Description: Discusses the use of cost analysis for evaluation. Topics include types of cost analysis and how and when to use them for evaluation. An annotated bibliography on the subject is also included.
Using Focus Groups for Evaluation
Author: Marczak, Mary (7 more by this author); Sewell, Meg
Description: Discusses the use of focus groups for evaluation. Topics include how and when to use focus groups for evaluation and the advantages and disadvantages of doing so. An annotated bibliography on the subject is also included.
Using Qualitative Research in Planning and Evaluating Extension Programs
Author: Guion, Lisa (2 more by this author); Flowers, Lamont
Description: This paper provides a brief overview of qualitative research and how it can be used in planning and evaluating Extension programs.
Using Structured Interviewing Techniques
Description: This guide is one of a series of papers issued by the Program Evaluation and Methodology Division (PEMD). The purpose of the series is to provide evaluators with guides to various aspects of audit and evaluation methodology, to illustrate applications, and to indicate where more detailed information is available.
Web Center for Social Research Methods
Description: This website is for people involved in applied social research and evaluation. In addition to my own work, you'll find lots of links to other locations on the Web that deal in applied social research methods. You'll also find comprehensive course resource centers for my courses at Cornell, previously published and unpublished papers, detailed examples of current research projects, useful tools for researchers (like a guide to selecting a statistical analysis), an extensive online textbook, a bulletin board for discussions, and more.
Author: Scheuren, Fritz
Description: This document describes the process of carrying out surveys and includes sections on designing, evaluating, and implementing questionnaires.
What is program evaluation? Module 1: What is Evaluation
Author: Shackman, Gene (7 more by this author)
Description: Provides a quick and basic background of what program evaluation is, and what a program evaluation process involves.
What’s the difference between N and n?
Author: Moore, Kristin Anderson (7 more by this author)
Description: This tip-sheet was created to demystify the difference between N and n. It demonstrates how to put the ideas to work when you report evaluation and needs assessment results.
– General Information –
Can You Call It a Focus Group?
Author: Kathlene, Larson (1 more by this author); Allen, Beverlyn Lundy; Grudens-Schuck, Nancy
Description: This article provides a framework for understanding and assessing the appropriate uses of focus groups.
Do's and Don'ts of Community Surveys
Author: Guy, Stan (1 more by this author)
Description: A brief fact sheet on basic tips for conducting surveys in the community. It includes basic and general guidlines involved in conducting community surveys.
Evaluation Challenges Module 1: Getting Answers
Author: Shackman, Gene (7 more by this author)
Description: How to answer questions concerning the outcomes of a program
Evaluation Challenges Module 2: Getting information useful to you
Author: Shackman, Gene (7 more by this author)
Description: Making use of people involved in the program to obtain information for the program evaluation
From Good Intentions to Effective Interventions: A Primer on Evidence-Based Prevention Programs
Author: Small, Stephen (3 more by this author)
Description: This presentation will provide an overview of evidence-based programs and practices for strengthening families and preventing youth problem behaviors. Participants will gain a deeper understanding of what it means for a program to be "evidence-based" and "cost-effective," as well as insights into how the accumulated evidence on effective programs can be used to improve existing programs and practices more generally. The presentation will conclude with a discussion of the advantages and drawbacks of using evidence-based programs and the role that Extension might play in their dissemination and use.
What is Program Evaluation? Module 2: The Evaluation Question
Author: Shackman, Gene (7 more by this author)
Description: Explains the helpfulness of using a program’s logic model ,or program theory, to develop evaluation questions.
What is Program Evaluation? Module 3: Methods
Author: Shackman, Gene (7 more by this author)
Description: Advantages and disadvantages to qualitative and quantitative evaluation methods.
What is Program Evaluation? Module 4: Surveys
Author: Shackman, Gene (7 more by this author)
Description: How and when to use surveys in a program evaluation
What is Program Evaluation? Module 5: Focus Groups
Author: Shackman, Gene (7 more by this author)
Description: How focus groups, key informant interviews, and observations may be used in a program evaluation.
What is Program Evaluation? Module 6: Did the program have an effect?
Author: Shackman, Gene (7 more by this author)
Description: Determining if a program caused an outcome
Who Are the Question-makers? A Participatory Evaluation Handbook
Description: This handbook provides some basic information on designing participatory evaluations and includes an overview of some of the essential steps involved.
– Other Resources –
Resource Spotlight: CYFAR Evaluation Community of Practice
Author: Nordby, Ann (21 more by this author)
Description: A new online community of practice brings the CYFERnet evaluation team and program staff involved in evaluation. Discussions expand on webinars and other resources from Extension and within CYFERnet.
– Professional Development –
Articulating Program Results: Using Logic Models to Identify Success
Author: Marek, Lydia (16 more by this author)
Description: Of primary importance in developing logic models is establishing and maintaining a focus on results rather than on activities. Logic models allow program professionals to clearly articulate what needs to be accomplished in their program. Understanding how a program achieves results is critical to determine if program elements should be modified and provide stakeholders with evidence that the program is effective. Logic models help program professionals, evaluators and stakeholders reach consensus about which elements are essential to the program and consequently program professionals and researchers are more able to identify faulty or implausible links early on.
Author: DeBord, Karen (61 more by this author); Lynn Borden
Description: This site focuses on understanding needs assessment and evaluation, and reporting the findings of a needs assessment or evaluation.
* Have you ever gotten frustrated with the evaluation process?
* Have you ever completed a report and not really told the WHOLE story of your educational efforts?
* Have you ever wondered how to take the data you have collected and tell the success stories of your programs?
* Have you ever found you had wonderful programs but could not communicate your successes to others?
If so, this interactive learning site will take you BEYOND the DATA and help you learn a high level of reporting and how to present your data in a meaningful way.
Evaluation and Intentionality: You Can't Have One Without the Other
Description: Intentional programming and program evaluation go hand in hand. Without intentionally identifying and programming towards targeted outcomes it becomes impossible to develop a pertinent evaluation. This session looks specifically at issues related to both topics including the development of logic models, program fidelity, the black box, and the design and implementation of effective evaluations. Participants will gain an understanding of the relationship between intentional programming and evaluation and have opportunities to apply this knowledge to their own programs.
Evaluation Workshops and Presentations
Description: This website contains links to evaluation trainings and presentations by the University of Wisconsin-Extension Cooperative Extension Program Development and Evaluation Branch.
Family Resiliency Interactive Program Evaluation Development Site
Author: DeBord, Karen (61 more by this author)
Description: Good evaluation for any program comes from good program planning. The Extension Parenting Evaluation Decision Framework is an interactive program planning tool that allows you to step through a total process. The total model includes 10 steps. You should plan about 20-30 minutes of time to work through all the steps. The authors of the original document and the creators of this interactive site hope you will find this a learning and thought provoking experience.
Resource Spotlight: CYFAR Evaluation Community of Practice
Author: Nordby, Ann (21 more by this author)
Description: A new online community of practice brings the CYFERnet evaluation team and program staff involved in evaluation. Discussions expand on webinars and other resources from Extension and within CYFERnet.
Using Multiple Methods to Demonstrate Program Impacts
Author: Lee, Thomas; Cox, Janet
Description: Increasingly, Extension educators are being asked to demonstrate that they are making a difference - that their programs are having an impact. We need to go beyond asking people if they liked the refreshments to finding out if they are doing things differently as a result of participating in Extension programs. This is an audiorecording and powerpoint presentation from an interactive teleconference that provides an overview of multiple evaluation methods, and explains the benefits and drawbacks of each type, and the value of using more than one method to have more confidence in the results that are achieved. Utah's Youth and Families with Promise program evaluation, recognized as an exemplary evaluation model by CYFAR, is explained to illustrate the use of multiple methods in a particular project.
– Programming Resource –
Choosing an evaluation approach – A review of design alternatives
Description: Kaye, E., & Graham, E. (2009). Choosing an evaluation approach – A review of design alternatives. National Child Welfare Evaluation Summit. This is a 37-slide powerpoint presentation which reviews the most common evaluation designs, critiques pros and cons as well as myths and misconceptions regarding those designs, and discusses programmatic and organizational factors that should be considered when selecting a design.
Do's and Don'ts of Community Surveys
Author: Guy, Stan (1 more by this author)
Description: A brief fact sheet on basic tips for conducting surveys in the community. It includes basic and general guidlines involved in conducting community surveys.
Description: This publication focuses on promising practices and strategies for evaluating child services; from the Harvard Family Research Project ( available online and in print).
Making a Difference: Needs Assessment for Coalitions: CDFS-9
Author: Archer, Thomas; McCaslin, N. L.; Cripe, Robert
Description: A fact sheet that discusses needs assessment, including how to plan and design a needs assessment and how to report and use the information gained.
Penn State - Program Evaluation Webpage
Description: Penn State, Cooperative Extension & Outreach provides links to evaluation information, resources, and toolkits.
What is program evaluation: A beginners guide
Description: This is a 17-page pamphlet designed to serve as a beginners guide, explaining in very basic terms what evaluation is, and then presenting a logic model example, followed by basic descriptions of various methods (e.g., surveys, analysis of administrative data, key informant interviews, observation, focus groups) and the advantages and disadvantages of each.
– Research –
Evaluation Science and Family Therapy
Author: Mancini, Jay (6 more by this author); Marek, Lydia I.; McCollum, E; Huebner, Angela J.
Description: This link is to a book chapter abstract published by members of the CYFERnet Evaluation Team that focuses on program evaluation science and family interventions. Contact information for obtaining the chapter is provided.
From Good Intentions to Effective Interventions: A Primer on Evidence-Based Prevention Programs
Author: Small, Stephen (3 more by this author)
Description: This presentation will provide an overview of evidence-based programs and practices for strengthening families and preventing youth problem behaviors. Participants will gain a deeper understanding of what it means for a program to be "evidence-based" and "cost-effective," as well as insights into how the accumulated evidence on effective programs can be used to improve existing programs and practices more generally. The presentation will conclude with a discussion of the advantages and drawbacks of using evidence-based programs and the role that Extension might play in their dissemination and use.
Issues in Evaluating Parenting Curricula
Author: Powell, Douglas
Description: This piece addresses some of the methodological flaws are prevalent and may contribute to the finding of limited program efficacy in parenting education curricula. The author shares lessons learned in terms of (1) incorporating evaluation into all phases of program development and implementation, including program design decisions; (2) employing a theoretical or conceptual model of the anticipated mechanisms of change in parenting programs; and (3) systematically examining for differential program effects in analysis' of program outcomes.