Conducting an Evaluation

Different types of data are described below:

• Quantitative data, such as water quality data, Web analytics, and epidemiology statistics.
• Qualitative data, such as customer comments, media reports, staff memos, etc.

See Appendix C: Online Resources, Data Management for links to information on qualitative data collection.

See the Advisory Feedback Guide [DOCX – 4 pages] and Call Center Data Collection Framework [DOCX – 2 pages] for collecting information from partners.

Surveys

Water systems should consider surveying their customers about the advisory. Surveys can be used to determine the effectiveness of the advisory and to determine whether the advisory influenced personal actions.

• Customer service surveys may include a section related to an advisory.
• Surveys can be designed to measure perceptions, actions, and communication effects and outcomes.
• The perceptions and recall of respondents will change with the amount of time that passes between an advisory and a survey.
• Surveys can be short and focused or longer to gather a full range of data.
• Online surveys offer a fast and efficient way to solicit and analyze feedback from a wide range of respondents.

Consult the Post-Advisory Community Survey [DOCX – 6 pages] or CASPER (Community Assessment for Public Health Emergency Response) for ideas about what to ask in a survey and for methods to use when conducting a survey.

Data Management

Manage data as it is collected. Data management activities may include:

• Storing data as hard and electronic copies in a central location for easy access.
• Requesting information from partners involved with the advisory.
• Asking partners about limitations or privacy policies that might limit the ability to report the data.
• Working with database experts to ensure that the database structure is optimal for doing the planned analysis.

Data sets are valuable when they are applied to improve future responses. The information collected should be analyzed, synthesized, and reported in a useful format. Use the data to develop questions for public outreach and for the basis of evaluation. Develop a plan for data analysis that will give the answers needed for improvement and decision making.

Appendix C: Online Resources, Data Management contains links to websites on data frameworks.

Table 2 presents examples of different types of resources and data that can be used to evaluate communication activities during a drinking water advisory.

Table 2: Examples of Types and Sources of Drinking Water Advisory Data