The Experiences of Eighteen Communities and
Progress Towards Inter-Agency Coordination to Reduce Alcohol and Substance
Abuse Among Adolescents Evaluation Report for the Washington State Incentive
Grant (July 1998 – July 2002)
Executive Summary
Washington State was awarded a nine million State
Incentive Grant (SIG) for the dual purposes of promoting prevention
system changes among state agencies and of implementing more evidence-based
prevention programs. The ultimate goal was to increase coordination
and efficiency and to improve outcomes of prevention activities so that
fewer youth would abuse alcohol and drugs.
Washington State visualized these system changes as occurring within
a framework of decentralized power, where better coordination would
follow from a common purpose and from a decision-making framework based
on better information. The system would still give local constituents
choices of local prevention foci, resource coordination, and program
selection, but within a common, outcome-based framework established
among diverse state agencies. State agency objectives within this framework
were to:
Agree on a common set of prevention goals and outcome measures:
Collect and report better community data based on a common risk and
protective factor framework, to help local prevention planners use
a more data driven planning model. Futhermore, agree on statewide
prevention goals and benchmarks that could partially guide local choices.
Develop common assessment tools of community needs and resources:
Create common and collaborative requirements for local need and resource
assessments and planning, to decrease duplication of effort at the
local level and improve outcomes.
IdentiFiscal Year selection criteria and a common set of science-based prevention
programs: Agree on selection criteria to identiFiscal Year science-based programs
that can best address the needs identified from common assessment
and measures, and facilitate their local implementation.
Develop a uniform reporting mechanism for participant outcomes:
Discuss data compatibility across agencies for community wide reporting,
and pilot the Everest Prevention Outcomes Evaluation Management System
to assess program impacts among participants.
Establish guidelines for leveraging and redirecting resources: Encourage
both specialized independent prevention efforts and resource coordination,
leveraging and redirection among community partners.
Create a cross-agency system of professional development: Create
a common training system for prevention PROFESSIONALs working for
different state agencies.
Summary Findings
At the state level, most prevention partners participated in creating
the plan for interagency coordination, considered it realistic and supported
it. Progress has been made in building the common prevention plan and
implementing the planned state objectives. Some major issues still remain
unresolved, but state agencies and community constituents have common
interests in resolving these issues in the long run.
Evaluation findings among selected Washington communities show that
science-based prevention can be learned and implemented, outcomes can
be measured, and there are significant possibilities for leveraging
community funds. It thus becomes strategically important for state agencies
interested in efficiencies, effective coordination and demonstrable
outcomes to promote such prevention efforts.
State Achievements
Significant progress was made on five of the six state-level objectives.
They were:
A common set of prevention goals and outcome measures: Agreements
were reached on statewide benchmarks, on measures for the Washington
State Survey of Adolescent Health Behavior, and on scales for monitoring
outcomes for SIG program participants. A common health and substance
use survey is being implemented in the Fall of 2002. Student participation
in the student survey has doubled. Furthermore, archival indictors
are being validated against student survey results.
Common assessment tools of community needs and resources: All
Washington counties conducted an inter-agency, collaborative needs
assessment in the spring of 2001.
The identification of science-based prevention programs: SIG communities
have used the programs listed on the Internet by the Western Center
for the Application of Prevention Technology (WestCAPT).
Uniform reporting mechanisms for participant outcomes: SIG communities
have piloted the Everest Prevention Outcomes Evaluation Management
System.
System of professional development: A training curriculum was developed
and four training sessions were implemented in 2001 by WestCAPT with
state agency funding; voluntary certification was available in June
2002.
The decision was made not to develop guidelines for leveraging and
redirecting money and resources (one of the six state objectives), but
to wait until the inter-agency group had experience in monitoring statewide
benchmarks
Strategy development for system changes in prevention was largely
successful by channeling input from many different constituents pushing
for more coordination; placing prevention system changes as part of
the Governor's agenda; avoiding inter-agency funding issues by
focusing on common statewide goals; and setting up a formal inter-agency
group to monitor statewide benchmarks.
State agency representatives gained inter-agency knowledge and strengthened
working relationships across agency boundaries by attending SIG meetings.
State-Level Challenges
Top managers of state agencies, in follow-up interviews, identified
the following issues as the most important ones to resolve for further
movement toward the prevention system vision developed during this grant.
How to gain long-term funding to support common, inter-agency prevention
databases available to communities, including scales to measure program
participant outcomes and community-level outcomes.
How to create and fund a system of collaborative, inter-agency technical
assistance and training.
How to deal with accountability of the inter-agency group that will
monitor the new substance abuse prevention system, and resolve issues
of communication between local and state agencies.
How to resolve the disagreements about whether the model of collaboration
for the substance abuse prevention system should stand alone or whether
it could be a model for other types of prevention.
How to conduct further research, and with what funds, to demonstrate
community wide outcomes in SIG sites when student survey data become
available in the Fall of 2002, 2004 and 2006. -- to provide evidence
for links between particular system change strategies in SIG sites
and better, long term, community wide
outcomes.
Community Achievements
Eighteen communities in Washington were awarded SIG grants based on
a competitive selection process. They agreed to strive to accomplish
five objectives:
Coordinate prevention planning with community partners,
Use a common ‘risk and protection factor' framework for
assessing community prevention needs and for targeting prevention
activities,
Consider relevant data on risk and protective factors in their planning,
Implement prevention programs shown to be effective using evidence-based
criteria (i.e. science-based programs), and
Collect outcomes among program participants in order to improve
program effectiveness.
All eighteen SIG communities achieved these objectives, even though
most had never implemented science-based programs, and 28 percent had
not used the risk and protective framework before. The use of a stepwise
logic model, outcome driven and data driven, was important in this achievement.
Almost all SIG communities were also able to coordinate, leverage and
redirect other resources among local partners, so as to increase and
institutionalize overall prevention efforts, which served more people
in the communities.
All SIG communities participated in the collection of outcomes among
program participants. Many used a new Internet based software developed
in Washington for this purpose: the Everest Prevention Outcomes Evaluation
Management System. The scales
used to measure relevant outcomes were found to be reliable. In many
cases, where programs were implemented with fidelity to the original
program design, expected changes occurred among program participants.
Whether these accomplishments (made mainly in the years 1999-2002) have
led to community-wide changes in the SIG sites awaits the analysis of
longer-term, trend data in student surveys and archival indicators.
Student survey and archival data are available for the initial period
of 1998, 1999 and 2000. SIG communities have committed to collecting
further survey data in their schools in the Fall of 2002, 2004 and 2006.
It is
hoped that further archival data will also be available at the necessary
level of small community geographies
Community Challenges
The availability and use of correct information is central to a more
science-based approach to prevention. Communities experienced challenges
that often related to limitations with existing data and how to interpret
and use data. SIG communities struggled and asked for help in the following
areas.
More technical assistance is needed in the selection of science-based
programs that would meet local needs and in training to overcome difficulties
in their implementation. Communities also need more training in the
use and interpretation of data useful for prevention planning and
monitoring program effectiveness.
Better and more local information for needs assessment and community
outcomes: Community level data on risk and protection profiles is
needed at more local, sub-county levels. Communities also need more
help in interpreting often-partial data. County-wide archival indicators,
based on data collected from various state agencies have been available
for many years. However, more work
needs to be accomplished to validate them against student survey information
and make them available at smaller geographies for prevention planning.
Better reporting on participant outcomes: More user-friendly, interpretable
reports on the outcomes of program participants.
Resolution of the remaining state challenges would fulfill important
community prevention needs by providing:
Better information on which to base community prevention decisions
and better data to demonstrate effectiveness of prevention efforts.
Common tools and training to facilitate collaborative planning and
reduce duplication.
Clearer, less contradictory mandates from diverse state agencies,
who are together accountable for better results.
Better integration of substance abuse prevention efforts with other
local prevention priorities.
Higher chance of obtaining both private and state funding for prevention
programs that have been shown by research to have better outcomes.
Download Community
Report
The report "Implementing Science Based
Prevention to Reduce Alcohol and Substance Abuse Among Adolescents:
The Experiences of Eighteen Communities and Progress Towards Inter-Agency
Coordination" can be downloaded in two parts: Part
1 (content, 892 KB, 4.43) and Part 2 (appendices, 850KB, 4.43A)
To view this Portable Document
Format (PDF) you may experience errors or unexpected behavior
while opening or reading the file you downloaded. Therefore, we
suggest that you always use
the latest version of the Adobe Acrobat Reader. Persons with
disabilities may call to request a paper copy
For more ways to get in touch with the Department of Social
and Health Services, go to
the DSHS Contact
Information Web page. Technical Site Comments: DSHS Webmaster.
Copyright 2004 Washington State Department of Social and Health Services.
Implementing
Science Based Prevention