20 October 2021 at 12:00AM
Evaluator Inclusion in Program Planning and Design
Hello PMI Phoenix!
My name is Matthew Gallagher, and I am a PhD Candidate at Arizona State University. I am also a professional program evaluator, which means my job is to collect and use quantitative and qualitative data to aid program leaders and their teams in developing knowledge about, making data-informed decision on, and managing their programs.
Matthew Gallagher (right) collaborating with program leaders on program evaluation activities in Dili, Timor Leste.
Program leadership’s decision to include an evaluator during the planning and design phase of their programs is the critical first step necessary for evaluators to provide the benefits associated with the discipline of evaluation. Within the evaluation field, federal legislation, including the recently passed Evidence Act of 2019, promotes the inclusion of evaluators in the planning and design phase of federal programs. Additionally, evaluation academics and practitioners advocate for this level of inclusion within all types of programs.
However, despite legislation and advocacy efforts, the evaluation literature depicts a situation where evaluators are not consistently included in the planning and design phase of programs due to numerous barriers, including: 1) program funding announcements which request the inclusion of evaluation activities in proposed programs, but do not require evidence of any collaboration with a program evaluator; 2) training programs that offer little or no training on the benefits of hiring an evaluator during the program planning and design phase. Overall, the literature portrays programs as plagued by rushed and/or symbolic evaluations that are of limited use for program management, thereby frustrating program leaders and demoralizing evaluation practitioners.
To date, academic studies have examined approaches on how to incorporate an evaluator into a program’s planning and design phase (Fitzpatrick, 1988) and multiple researchers have advocated on behalf of the inclusion of the evaluator throughout all the phases of the project cycle (see: Patton, 1978; Stufflebeam, 2001; Preskill & Torres 2001; Mark, 2012; Scheirer, 2012). However, no studies have examined the extent to which evaluator inclusion occurs, or explored the reasons why a program leader decides whether to include an evaluator in a program’s planning and design phase. My PhD research aims to study this topic from the perspectives of both program evaluators and program leaders (who include program directors and program managers).
From the program evaluation side, I am currently collaborating with the American Evaluation Association to procure perspectives on this topic from 1,000 of their members, who were selected at random. To capture the perspectives of program leaders on this topic, I am collaborating with local chapters of the Project Management Institute. I have developed a questionnaire to collect your perspectives, and I hope I can count on your participation.
The new knowledge generated from this study has the potential to affect how program leaders and program evaluators are trained. When we have a mutual understanding of the extent to which evaluators are not included in the program planning and design phase, as well as what influences program leaders’ decisions to include or exclude evaluators during this phase, then we can develop a roadmap for how training content should be augmented to meet the federal mandate, fulfill advocacy efforts, and be proactive on exploring the topic more broadly.
If you are interested in providing your viewpoint on the topic of evaluator inclusion in the program planning and design phase, please click on the survey link below. The survey will take approximately 10 minutes to complete, and will be open until November 15th. Information about the study, your voluntary participation, and data privacy are on the first page of the link before you enter the survey. Thank you in advance for your thoughtful and honest responses!
Take the Survey Here:
