Measuring Safety Measures
A new toolkit for local agencies to evaluate their interventions
 

For the first time anywhere, a standardized, statistically robust evaluation tool is being made available to local recipients of traffic safety grants so that they can measure their programs' effectiveness, thanks to a project funded by the California Office of Traffic Safety (OTS) and implemented by the UC Berkeley Traffic Safety Center.

Researchers at the Center, with help from a team of traffic safety professionals around the state, are in the process of finalizing three manuals that show staff at local agencies how to conduct surveys that will accurately measure the effectiveness of programs that aim to increase the use of child passenger safety seats, seat belts and bicycle helmets. The survey results will also help OTS better identify strategies that work, and more accurately pinpoint locales in the state where programs are needed most.

The survey package was prompted by several needs.  Many local agencies do not have the funds to support evaluation efforts. Even if they do, many do not have staff members with the necessary expertise in statistics and program evaluation to produce usable, meaningful results. Finally, when local agencies do conduct evaluations, their methods are not always consistent with one another, which makes comparing results difficult, if not impossible. In order to get the best overall understanding of statewide usage rates, a more standard survey is required. The local use of standardized tools is expected to enhance occupant safety within the state, and the protocol for the evaluation will provide an important model for other studies (such as misuse of child restraints, for example). 

---

"Community or city-wide surveys are important for understanding what use looks like on the local level. With this information we will be able to target those areas for interventions."--Jill Cooper

---

"Currently our grantees mainly use crash data to evaluate effectiveness. This standardized protocol will give more credibility to occupant restraint and bicycle helmet surveys,” reports Chris Murphy, Deputy Director of OTS. “It is critical that we use those measurements to evaluate the programs that we fund and to be able to share the successes of the programs with other interested traffic safety professionals and organizations."

Many states, including California, have developed methodologies for evaluation at the statewide level, but none has created a survey protocol such as this that can be used on a local or city level. Jill Cooper, Program Manager of the Traffic Safety Center, describes the importance of that distinction. "Under the current system, data on smaller areas and cities are not available. The statewide survey is critical for capturing the average rate of seatbelt use for the state.  Community or city-wide surveys are important for understanding what use looks like on the local level. With this information we will be able to target those areas for interventions."

The survey kits show the user how to construct a protocol that establishes a baseline rate of compliance by means of a "before" survey. It includes instructions on how to physically organize the survey (usually in teams of two or more observers at a curbside location), and how to choose a location and time. Also included is an explanation of how to conduct a follow-up survey to see if usage has increased after the intervention-for example, a child safety seat fair at day care center. It also provides the forms for collecting data, and matrices and formulas for determining how many observations will be needed and ways to calculate the statistical certainty of a specific survey's results. 

"It is imperative that the programs we fund show results," says Murphy. "Often the question comes up 'Do your programs work, and how do you know?" This type of standardized survey will be effective at showing how successful such interventions are. Also, as Murphy explains, "Grants that can show measurable results are the types of programs that we would encourage other agencies to adopt."

The kits will be incorporated into the support materials that OTS provides all recipients of traffic safety grants and will be posted on the OTS Web site for downloading.

Murphy concludes, "To be the first state that I know of to complete this task is really gratifying, and I'm thrilled that it will be something that will have the UC Berkeley name on it to add to its credibility"

 

---

 

Traffic Safety Center Home