Program Evaluation

The Vermont Department of Health has a strong, ongoing commitment to high quality program evaluation. The structure for evaluation within the Division of Substance Use Programs (DSU) allows for rapid development of evaluation plans and launch of evaluation processes with either internal staff or external evaluators. All evaluation work is overseen by the division's internal evaluation staff. This serves to help coordinate evaluation resources, promotes consistent evaluation practices and builds in ongoing support and technical assistance for all evaluation activities.

Evaluation involves systematic collection of information about a program's activities, characteristics and outcomes that allows for:

  • Informing program development
  • Assessment of a program's strengths, challenges and opportunities
  • Improving program effectiveness
Evaluation Process
0-6 Months:  Implementation6-12 Months:  Program start up6-18 Months:  Begin to provide services12-24 Months:  Continuous Quality Improvement (QI)24+ Months:  Continued programming
  • RFP & Grantee Selection
  • Grants issued
  • Data informed program deliverables
  • Grantee planning and training
  • Define and set up programming & data collection
  • Evaluation planning
  • Data collection begins
  • Goal: Knowledge & attitudes shift
  • Data collection continues
  • Identify areas for continuous quality improvement
  • Goal: Behavior change
  • Evaluation continues
  • Statewide data may show change for large scale efforts
Population-level Change

When can we see population-level behavior change?

Population-level change requires long-term and sustained strategies, funding and legislation.

Image
Timeline showing smoking rates in the US and health milestones that eventually resulted in a decline in smoking rates.
Example of the time and complex strategies needed to see change in health behaviors at the population-level. (Click image to enlarge.)

Long-term success is built on incremental change. For example, the U.S. Department of Health and Human Services outlines the numerous smoking and health milestones over 130 years that contributed to the incline and decline in adult cigarette consumption. Strategies impacting the decrease in consumption varied in scope and included health messaging like the 1964 Surgeon General's report on smoking linking smoking and cancer and health and warning label requirements, legislative action like advertising bans and cigarette taxes and cessation initiatives like nicotine replacement gum and patches and other quit resources. 

A single strategy may have negligible change on a population-level, but in combination with a number of strategies from a variety of angles over decades can result in significant change.

What can program evaluations tell us?

Program evaluations tells us what parts of the program work well and what to change so that the program works better.

When can we see short-term behavior change?

It takes time from when funding is allocated to when new programs are implemented. Evaluation resources are also limited and must be prioritized. Depending on the timeline, evaluations may show short-term impacts on the people directly served by the program.

Contact Us
Division of Substance Use Programs (DSU)

Vermont Department of Health
280 State Drive
Waterbury, VT 05671-8340

802-651-1550
[email protected]

Last Updated: