Practical Research-based Information from the Nation's Leading Universities
Evaluation Design and Methods
Number of unique resources found: 77
Did you find what you were looking for? If not, try our full text search or let us know by completing our survey form.
Description: This resource website focuses on qualitative methodologies, particularly action research.
– Evaluation –
A Ranking Question for a Needs Assessment 
Author: Kiernan, Nancy
Description: This tip-sheet examines the use of ranking when conducting a needs assessment.
Action Research Electronic Reader
Author: Hughes, Ian ; Bob Dick
Description: This website provides an introduction to action research and discusses specific methods, e.g., observing, diary, focus groups, etc. There are also links to several related articles and resources focusing on action research.
Adapting Evaluation Measures for Hard to Reach Audiences
Author: Earthman, Erik; Marczak, Mary S.; Peterson, Donna J.; Richmond, Lucinda S.
Description: We are frequently unable to obtain
evaluation information from participants with limited
English or low literacy skills
young children, and people from non-majority
cultures. Together these groups represent the ‘hard to reach’ audiences of
evaluation. This bulletin
provides practical guidelines for program evaluators
working with ‘hard to reach’ audiences.
Alternative Methods for Collecting Evaluation Data
Description: Collection of documents discussing the use of non-traditional methods for the collection of evaluation data. Topics covered include: focus groups, cost analysis, portfolio assessment, qualitative interviews, and existing records.
An Educator's Guide to Evaluating The Use of Technology in Schools and Classrooms
Author: Quiñones, Sherri ; Rita Kirshstein
Description: This guide is designed to help you through the technology program evaluation process and to make it as painless as possible. Primarily geared to educators or administrators with little or no research experience, it is intended for use at the district or school level.
An Overview: Designing Initiative Evaluation
Author: Parsons, Beverly
Description: This resource is designed for external evaluators who conduct initiative evaluations for the W.K. Kellogg Foundation (WKKF). It presents a systems-oriented framework and four general designs for initiative and cluster evaluation.
Description: This guide to technology program evaluation has been developed into a full suite of resources, tools, and professional development activities designed to assist program and project leaders in collecting and analyzing data they can use in making decisions about the implementation of their projects. The framework's elements can be easily adapted for the evaluation of other types of grants and school improvement initiatives.
CYFERnet Exemplary Program Evaluations
Author: Peterson, Donna; Mancini, Jay A.; Marek, Lydia I.; Betts, Sherry C.
Description: Extension's capacity to evaluate community-based programs has increased dramatically. In 2001, the increased capacity was documented and celebrated. CYFERnet-Evaluation collaborators at the University of Arizona and Virginia Tech solicited nominations of State Strengthening and New Communities Projects that conducted exemplary program evaluations.
Developing a Logic Model: Teaching and Training Guide
Author: Taylor-Powell, Ellen; Ellen Henert
Description: A new resource for those teaching or mentoring community groups, Extension colleagues or others in creating and using logic models. The Guide includes ice breakers, FAQs and 17 hands-on activities with printable handouts for basic understanding and development of logic models. There is also a separate power point file to complement the guide.
Developing Process Evaluation Questions
Description: This brief provides criteria and examples to help in the development of process evaluation questions.
Description: This brief discusses what logic models are, types of logic models, how to read them, and steps to develop them.
Digital Storytelling: A Picture's Worth a Thousand Words: Evaluating Using Photographs
Author: Hurlbert, Debbie; Becky White
Description: The idea of using pictures and video is not a new one in evaluation, but emerging technologies are making it easier and more practical for evaluators to include these resources. Participants will learn about technologies that make it easy to gather and manipulate pictures and video to enhance the quality and impact of an evaluation.
Enhancing Program Performance with Logic Models
Description: This website links to an on-line course about the use of logic models in program evaluation.
Evaluating HIV Education Programs
Author: Popham, W.
Description: Five key guidelines are described to provide a step-by-step model for the evaluation of an HIV education program.
Description: This publication focuses on promising practices and strategies for evaluating child services; from the Harvard Family Research Project ( available online and in print).
Evaluation Made Easy: Examples of Practical Evaluations - Using Timelines
Author: Jakes, Susan; Garcia, Aleta; Peterson, Donna J.; Cassidy, Lisa
Description: This bulletin includes a brief introduction to a key indicator, a case example of a community-based program making a difference in the indicator highlighted, sample program goals and objectives, description of program intervention, a sample evaluation plan, key evaluation results, reporting hints and other resources.
Evaluation of HIV Prevention Programs using Qualitative Methods
Description: The three most commonly used qualitative methods are presented. These techniques can provide valuable information about program implementation, participant satisfaction, evidence of program effectiveness, and the identification of areas needing improvement. Qualitative measures can play an important role in an overall program evaluation effort.
Evaluation of Parenting Education Programs: A Parenting Evaluation Decision Framework
Author: DeBord, Karen; Wally Goddard; Robert Fetsch; Wendy Stivers
Description: As part of the National Network for Family Resiliency, a parenting education special interest group (SIG) was organized in 1996 to examine evaluation models for Extension Educators across the nation. The result is the Parenting Evaluation Decision Framework. The framework includes: Needs and asset assessment; Issue identification; Goal writing; Objective writing; Program intervention; Assessing outcomes; Reporting results; and Redesigning program based on evaluation results.
Author: Grudens-Schuck, Nancy; Lundy-Allen, Beverlyn; Larson, Kathlene
Description: This document briefly describes some fundamental aspects of focus groups, and distinguishes them from other familiar research tools.
From Good Intentions to Effective Interventions: A Primer on Evidence-Based Prevention Programs
Author: Small, Stephen
Description: This presentation will provide an overview of evidence-based programs and practices for strengthening families and preventing youth problem behaviors. Participants will gain a deeper understanding of what it means for a program to be "evidence-based" and "cost-effective," as well as insights into how the accumulated evidence on effective programs can be used to improve existing programs and practices more generally. The presentation will conclude with a discussion of the advantages and drawbacks of using evidence-based programs and the role that Extension might play in their dissemination and use.
Author: Chinman, Matthew; Pamela Imm; Abraham Wandersman
Description: This guide was created by a team of substance
abuse prevention researchers dedicated to helping community organizations reach positive outcomes through a process of
answering ten questions that contain all the elements needed for successful planning, implementation, and evaluation (PIE).
This manual represents a collaborative effort to synthesize evidence-based knowledge and translate it into evidence-based practice.
Author: Chinman, Matthew; Pamela Imm; Abraham Wandersman
Description: This document contains the tools for Getting To Outcomes, including examples for needs assessments, focus groups, process evaluations, data collection and analysis, satisfaction surveys, etc.
Description: This brief provides a checklist and examples to help assess program goals and objectives.
How Can I Assess the Quality of My Program? Tools for Out-of-School Time Program Practitioners
Author: Kahn, Jordan ; Christina Theokas, Ph.D.; Jacinta Bronte-Tinkew, Ph.D.
Description: This brief identifies some of the advantages of using program quality assessment tools that have been developed for self-assessment and program improvement, and suggests a number of quality assessment tools currently available to assess program quality.
Integrating Logic Models and Work Plans
Description: This brief will define logic models and work plans and the interrelationship between the two; suggest how programs can visually link these two; and explain why linking them is beneficial.
Issues in Evaluating Parenting Curricula
Author: Powell, Douglas
Description: This piece addresses some of the methodological flaws are prevalent and may contribute to the finding of limited program efficacy in parenting education curricula. The author shares lessons learned in terms of (1) incorporating evaluation into all phases of program development and implementation, including program design decisions; (2) employing a theoretical or conceptual model of the anticipated mechanisms of change in parenting programs; and (3) systematically examining for differential program effects in analysis' of program outcomes.
Description: This brief illustrates the basics of logic model creation and implementation.
On-Line Evaluation Resource Library
Description: This resource was developed for professionals seeking to design, conduct, document, or review project evaluations. It makes available evaluation plans, instruments, and reports for National Science Foundation projects that can be used as examples.
Online Evaluation Resource Library
Description: The Online Evaluation Resource Library (OERL) is meant for professionals seeking to design, conduct, document, or review project evaluations. OERL is funded by the National Science Foundation.
Outcome Evaluation: A Guide for Out-of-School Time Practitioners
Author: Allen, Tiffany; Jacinta Bronte-Tinkew
Description: This resource provides a basic review of outcome evaluations, discusses why they are important and when they are useful, and discusses ways they can be implemented in out-of-school time programs. It also highlights the experience of an out-of-school time program that has conducted an outcome evaluation and provides a list of helpful evaluation resources.
Description: This guide provides an overview of surveys in social science research. It discusses the many kinds of surveys and provides examples. It also includes information on the process of designing and conducting surveys, and analyzing and reporting on survey results.
Parental use of Child Care: A Guide for Constructing Parent Surveys
Author: Emlen, Arthur; Roberta Weber
Description: This is a guide for constructing surveys on factors driving parental use of supplemental child care. It includes discussions of constructs and variables related to household, parent, work, and child characteristics, child care arrangements, and parent choice.
Practical Guide to Monitoring and Evaluation of Rural Development Projects
Description: This resource provides information on the development, installation and use of effective project-level monitoring and evaluation systems as tools for impact-oriented management of rural development projects.
Preparing for a Collaborative Community Assessment
Description: A fact sheet that discusses issues related to collborative community assessments. It highlights characteristics of successful collaborative community assessments, steps involved, and what to do with the results of the assessment.
Probing Questions in Interviews
Description: This short brief offers several practical suggestions for using probing questions when conducting qualitative research through interviews.
Process Evaluations: A Guide for Out-of-School Time Practitioners
Author: Bowie, Lillian; Jacinta Bronte-Tinkew
Description: This resource describes why process evaluations are important, when they should be used, and how they can be a useful tool for out-of-school time program practitioners. It also provides guidelines and strategies for implementing process evaluations and some concrete examples that illustrate their usefulness. It concludes with a list of helpful resources for out-of-school time program practitioners.
Author: Boeree, C. George
Description: This resource was created for Students/Faculty in introductory qualitative methods courses. It provides a broad look at qualitative methods including phenomenology, structural analysis,observation and interviewing.
Quasi-Experimental Evaluations
Author: Moore, Kristin Anderson
Description: This brief describes the varied types of quasi-experimental evaluations, what can be learned from such studies, and when it is appropriate to conduct them.
Questionnaire Design and Analysis Workbook
Author: Galloway, Alison
Description: This online workbook provides a hands-on approach to learning questionnaire design.
Questionnaire Design: Asking Questions with a Purpose
Author: Taylor-Powell, Ellen
Description: This resource goes step-by-step through the process of creating an effective questionnaire.
Random Assignment Evaluation Studies
Author: Moore, Kristin Anderson ; Allison Metz, Ph.D.
Description: Random assignment experimental studies involve using a "lottery" system to randomly assign participants to either a treatment group that receives program services or a control group that does not receive these services, and then comparing outcomes for the two groups. Only rigorous experimental studies can definitively establish that a program causes changes in outcomes for children and youth. This brief explains why random assignment studies are important, how to conduct them, and addresses reservations that program providers might have.
Reflective Assessments: A Tool for Learning
Author: Edwards, Sharon; Cutler, Ira
Description: This article describes "reflective assessment," a method organizations can use to measure program impact.
Research Methods Knowledge Base
Author: Trochim, William
Description: This resource provides a comprehensive introduction to social science research methods. It covers the entire research process including: formulating research questions, sampling, measurement, research design, data analysis, and writing a research paper.
Sample Size for Community Needs and Resource Assessment Surveys
Description: A fact sheet that discusses issues concerning sample size for community surveys.
Sampling and Surveying Handbook
Description: This resource provides a broadly applicable introduction to designing surveys or sampling for surveys.
Author: Galloway, Alison
Description: This resource provides a hands-on approach to learn sampling methods.
The 2002 User Friendly Handbook for Project Evaluation
Author: Frechtling, Joy
Description: This publication serves as a guide for evaluating the National Science Foundation's educational programs. It discusses both quantitative and qualitative evaluation methods.
The Use of Portfolio Assessment in Evaluation
Author: Sewell, Meg; Horn, Melanie; Marczak, Mary
Description: Discusses the use of portfolio assessment for evaluation. Topics include how and when to use portfolios for evaluation, and the advantages and disadvantages of doing so. Also includes an annotated bibliography on the subject.
The Use of Qualitative Interviews in Evaluation
Author: Sewell, Meg
Description: Discusses the use of qualitative interviewing for evaluation. Topics include types of qualitative interviewing and how and when to use them for evaluation. An annotated bilbliography on the subject is also included.
User's Guide to Evaluation for National Service Programs
Description: This evaluation guide is is widely applicable to beginners conducting internal evaluations of a service program. The guide is designed to assist programs by addressing four key priority areas: education, public safety, human needs, and environment.
Using an Evaluation for a Needs Assessment
Author: Kiernan, Nancy
Description: This tip-sheet discusses the challenges and benefits of using an evaluation for a needs assessment.
Using Cost Analysis in Evaluation
Author: Sewell, Meg; Marczak, Mary
Description: Discusses the use of cost analysis for evaluation. Topics include types of cost analysis and how and when to use them for evaluation. An annotated bibliography on the subject is also included.
Using Focus Groups for Evaluation
Author: Marczak, Mary; Sewell, Meg
Description: Discusses the use of focus groups for evaluation. Topics include how and when to use focus groups for evaluation and the advantages and disadvantages of doing so. An annotated bibliography on the subject is also included.
Using Qualitative Research in Planning and Evaluating Extension Programs
Author: Guion, Lisa; Flowers, Lamont
Description: This paper provides a brief overview of qualitative research and how it can be used in planning and evaluating Extension programs.
Using Structured Interviewing Techniques
Description: This guide is one of a series of papers issued by the Program Evaluation and Methodology Division (PEMD). The purpose of the series is to provide evaluators with guides to various aspects of audit and evaluation methodology, to illustrate applications, and to indicate where more detailed information is available.
Web Center for Social Research Methods
Description: This website is for people involved in applied social research and evaluation. In addition to my own work, you'll find lots of links to other locations on the Web that deal in applied social research methods. You'll also find comprehensive course resource centers for my courses at Cornell, previously published and unpublished papers, detailed examples of current research projects, useful tools for researchers (like a guide to selecting a statistical analysis), an extensive online textbook, a bulletin board for discussions, and more.
Author: Scheuren, Fritz
Description: This document describes the process of carrying out surveys and includes sections on designing, evaluating, and implementing questionnaires.
What is program evaluation? Module 1: What is Evaluation
Author: Shackman, Gene
Description: Provides a quick and basic background of what program evaluation is, and what a program evaluation process involves.
What’s the difference between N and n?
Author: Moore, Kristin Anderson
Description: This tip-sheet was created to demystify the difference between N and n. It demonstrates how to put the ideas to work when you report evaluation and needs assessment results.
– General Information –
Can You Call It a Focus Group?
Author: Kathlene, Larson; Allen, Beverlyn Lundy; Grudens-Schuck, Nancy
Description: This article provides a framework for understanding and assessing the appropriate uses of focus groups.
Digital Storytelling: A Picture's Worth a Thousand Words: Evaluating Using Photographs
Author: Hurlbert, Debbie; Becky White
Description: The idea of using pictures and video is not a new one in evaluation, but emerging technologies are making it easier and more practical for evaluators to include these resources. Participants will learn about technologies that make it easy to gather and manipulate pictures and video to enhance the quality and impact of an evaluation.
Evaluation Challenges Module 1: Getting Answers
Author: Shackman, Gene
Description: How to answer questions concerning the outcomes of a program
Evaluation Challenges Module 2: Getting information useful to you
Author: Shackman, Gene
Description: Making use of people involved in the program to obtain information for the program evaluation
From Good Intentions to Effective Interventions: A Primer on Evidence-Based Prevention Programs
Author: Small, Stephen
Description: This presentation will provide an overview of evidence-based programs and practices for strengthening families and preventing youth problem behaviors. Participants will gain a deeper understanding of what it means for a program to be "evidence-based" and "cost-effective," as well as insights into how the accumulated evidence on effective programs can be used to improve existing programs and practices more generally. The presentation will conclude with a discussion of the advantages and drawbacks of using evidence-based programs and the role that Extension might play in their dissemination and use.
Preparing for a Collaborative Community Assessment
Description: A fact sheet that discusses issues related to collborative community assessments. It highlights characteristics of successful collaborative community assessments, steps involved, and what to do with the results of the assessment.
What is Program Evaluation? Module 2: The Evaluation Question
Author: Shackman, Gene
Description: Explains the helpfulness of using a program’s logic model ,or program theory, to develop evaluation questions.
What is Program Evaluation? Module 3: Methods
Author: Shackman, Gene
Description: Advantages and disadvantages to qualitative and quantitative evaluation methods.
What is Program Evaluation? Module 4: Surveys
Author: Shackman, Gene
Description: How and when to use surveys in a program evaluation
What is Program Evaluation? Module 5: Focus Groups
Author: Shackman, Gene
Description: How focus groups, key informant interviews, and observations may be used in a program evaluation.
What is Program Evaluation? Module 6: Did the program have an effect?
Author: Shackman, Gene
Description: Determining if a program caused an outcome
Who Are the Question-makers? A Participatory Evaluation Handbook
Description: This handbook provides some basic information on designing participatory evaluations and includes an overview of some of the essential steps involved.
– Other Resources –
Resource Spotlight: CYFAR Evaluation Community of Practice
Author: Nordby, Ann
Description: A new online community of practice brings the CYFERnet evaluation team and program staff involved in evaluation. Discussions expand on webinars and other resources from Extension and within CYFERnet.
– Professional Development –
Articulating Program Results: Using Logic Models to Identify Success
Author: Marek, Lydia
Description: Of primary importance in developing logic models is establishing and maintaining a focus on results rather than on activities. Logic models allow program professionals to clearly articulate what needs to be accomplished in their program. Understanding how a program achieves results is critical to determine if program elements should be modified and provide stakeholders with evidence that the program is effective. Logic models help program professionals, evaluators and stakeholders reach consensus about which elements are essential to the program and consequently program professionals and researchers are more able to identify faulty or implausible links early on.
Author: DeBord, Karen; Lynn Borden
Description: This site focuses on understanding needs assessment and evaluation, and reporting the findings of a needs assessment or evaluation.
* Have you ever gotten frustrated with the evaluation process?
* Have you ever completed a report and not really told the WHOLE story of your educational efforts?
* Have you ever wondered how to take the data you have collected and tell the success stories of your programs?
* Have you ever found you had wonderful programs but could not communicate your successes to others?
If so, this interactive learning site will take you BEYOND the DATA and help you learn a high level of reporting and how to present your data in a meaningful way.
Evaluation Workshops and Presentations
Description: This website contains links to evaluation trainings and presentations by the University of Wisconsin-Extension Cooperative Extension Program Development and Evaluation Branch.
Family Resiliency Interactive Program Evaluation Development Site
Author: DeBord, Karen
Description: Good evaluation for any program comes from good program planning. The Extension Parenting Evaluation Decision Framework is an interactive program planning tool that allows you to step through a total process. The total model includes 10 steps. You should plan about 20-30 minutes of time to work through all the steps. The authors of the original document and the creators of this interactive site hope you will find this a learning and thought provoking experience.
Focus on Research Designs: Improving Your Child- and Youth-Focused Research
Author: Hair, Elizabeth; Lina Guzman
Description: This presentation will assist new researchers and practitioners in how to navigate the pitfalls and take advantage of resources as they design and implement social science research studies with a child and youth focus. Attendees will learn about designing a good research project, issues surrounding measurement and psychometrics, issues surrounding established data, and questionnaire and survey design.
Using Multiple Methods to Demonstrate Program Impacts
Author: Lee, Thomas; Cox, Janet
Description: Increasingly, Extension educators are being asked to demonstrate that they are making a difference - that their programs are having an impact. We need to go beyond asking people if they liked the refreshments to finding out if they are doing things differently as a result of participating in Extension programs. This is an audiorecording and powerpoint presentation from an interactive teleconference that provides an overview of multiple evaluation methods, and explains the benefits and drawbacks of each type, and the value of using more than one method to have more confidence in the results that are achieved. Utah's Youth and Families with Promise program evaluation, recognized as an exemplary evaluation model by CYFAR, is explained to illustrate the use of multiple methods in a particular project.
– Programming Resource –
Description: This publication focuses on promising practices and strategies for evaluating child services; from the Harvard Family Research Project ( available online and in print).
– Research –
Evaluation Science and Family Therapy
Author: Mancini, Jay; Marek, Lydia I.; McCollum, E; Huebner, Angela J.
Description: This link is to a book chapter abstract published by members of the CYFERnet Evaluation Team that focuses on program evaluation science and family interventions. Contact information for obtaining the chapter is provided.
From Good Intentions to Effective Interventions: A Primer on Evidence-Based Prevention Programs
Author: Small, Stephen
Description: This presentation will provide an overview of evidence-based programs and practices for strengthening families and preventing youth problem behaviors. Participants will gain a deeper understanding of what it means for a program to be "evidence-based" and "cost-effective," as well as insights into how the accumulated evidence on effective programs can be used to improve existing programs and practices more generally. The presentation will conclude with a discussion of the advantages and drawbacks of using evidence-based programs and the role that Extension might play in their dissemination and use.
Issues in Evaluating Parenting Curricula
Author: Powell, Douglas
Description: This piece addresses some of the methodological flaws are prevalent and may contribute to the finding of limited program efficacy in parenting education curricula. The author shares lessons learned in terms of (1) incorporating evaluation into all phases of program development and implementation, including program design decisions; (2) employing a theoretical or conceptual model of the anticipated mechanisms of change in parenting programs; and (3) systematically examining for differential program effects in analysis' of program outcomes.