Program Evaluation

The road to success starts with good program planning. Good program planning starts with having a skilled program evaluator involved from the beginning, even before the program is designed. A talented program evaluator can work with you to develop your roadmap to success.

Doing program evaluation is the only way to determine whether your investments (or the investments of others) are yielding desirable outcomes. Program evaluation is a systematic way of collecting, analyzing, and using data to answer questions about a program, service, or policy.

Some of the questions that evaluation answers include:

1) Did we reach our priority population as we had hoped?

2) Was the program implemented as we had planned?

3) Did the program meet its own goals and objectives?

4) Did it have a positive impact on the lives of participants?

5) What worked and what did not?

6) How do we know?

7) How should we improve the program?

The 1795 Method of Program Evaluation

To evaluate your program, we first must know it well. Therefore, we take the time and effort to learn about your program in detail. To do so, we typically do the following:

Explore and Discover

· Face-to-face interviews with Program Director & staff

· Identify the program’s mission, goals, objectives, activities, and evaluation plan

· Obtain copies of the manual, policies and procedures manual, curricula, past evaluation reports, evaluation methods, and past data collection tools

Conduct an In-Depth Content Analysis

· Identify the program’s design history and elements, including theoretical foundations

· Compare and contrast the current program design with published best practices

· Evaluate the strength of the program’s written goals, objectives, and activities

· Assess the fidelity of implementation

· Confirm accuracy of our content analysis with the Program Director and staff

Evaluate the Quality of Past Evaluation Methods and Tools

· Assess the alignment among the program’s mission, goals, objectives, and activities

· Identify and evaluate linkages between program objectives and evaluation

· Determine if existing evaluation methods allow objectives to be measured

· Compare current/past evaluation methods and tools to best practices

Collect and Analyze Data

· Obtain existing data that were collected in the past via existing tools

· Obtain original data via tools that we design (when existing original data do not exist)

· Enter data into statistical analysis software

· Analyze data based on criteria of success within the program’s objectives

· Convert statistical analysis output to tabular and text format

Our Unique Structural-Design Method

We use a proprietary structural design process focused on four aspects of your program:

1) The Internal Infrastructure of the Program: First, we carefully identify the program’s internal design infrastructure and assess the degree to which the program’s design matches recommended best practices in the published research.

2) The Degree of Alignment Among Program Components: Next, we measure the degree of alignment among all program components.

3) The Degree to Which the Program Met Its Stated Objectives: Then, we carefully assess the degree to which each objective was met by the program.

4) The Degree to Which the Program Impacted the Priority Population: Lastly, we analyze how members of the priority population were changed in positive ways by the program.

Your Destination

In closing, it is very important that you ponder these questions: Are you currently on a road that will lead to your desired destination? What outcomes do you desire for your program? Will the road that you currently travel and the data that you are collecting help you achieve those outcomes? Can you prove to your stakeholders that your program is working and improving lives?

Increased Accountability for Outcomes

In this age of increased fiscal accountability, tight budgets, and intense competition for grants, it is vital that you prove that your program works and improves the lives of program participants. No longer can administrators and program planners use weak measures such as attendance, number of brochures distributed, hits to a website, clients served, or the number of events completed. That weak type of data proves nothing about program impact! To be competitive today, you must have high quality, outcomes data.

Perhaps your funders, grantors, board members, and stake holders are demanding stronger and more rigorous outcomes from your programs. Perhaps you would like to apply for larger grants, but you cannot prove to others that your program works. If your current system of evaluation cannot provide you with strong evidence that your program is yielding a positive return on investment, then your program may be at risk.

Call or email us today. We have the expertise and experience to help you.

Other Areas of Expertise: