Frequently Asked Questions

What is the Science Impact Framework?

It is a simple approach (method) for tracing and linking CDC science to events and/or actions with recognized influence beyond the science of citations. We chose to go with science rather than research so we can capture other CDC efforts such as developing guidelines and recommendations, which contribute to health outcomes.

Why was this framework developed?

The framework was developed to be able to measure impact beyond citation data. There has been heavy dependence on journal metrics. However, metrics show the reach of the research in terms of how widely it is disseminated and the uptake. But, journal metrics do not characterize the influence created, such as resulting actions or changes or the manner in which the research knowledge is used.

How was the Science Impact Framework developed?

In developing the Science Impact Framework, several other frameworks were studied. This framework is an extension of the Institute of Medicine’s (IOM) “Degrees of Impact” Framework. Also, frameworks from the UK, UNDP, and others shaped the Science Impact Framework. The IOM model suggests an incremental progression of processes and actions. IOM services in an advisory role. So, the scope of their work is in the realm of knowledge diffusion (user- pull end of the spectrum). CDC has a broader scope; diffusion of knowledge, technology creation, capacity building, and program/initiative implementation. The key attraction for us with the IOM model is the focus on influences. We defined what the influences might be based on our scope of work at CDC.

 Top of Page

How does the framework work?

The framework tells us to begin with a point of scientific significance and then to identify forward and/or backward links from the significant event in order to trace the influence of CDC science. It gauges broader societal, environmental, cultural and economic impact using a combination of narrative and quantitative and qualitative indicators. The ultimate goal for public health is health outcomes. However, it takes time for the impact of a body of work to be apparent. Key is finding those short term indicators that are predictive of long term impact. In tracing the link, there is more emphasis on contribution rather than attribution (it is difficult to assign credit to any single entity). There is also a focus on the manner in which the scientific or research knowledge is being used.

The five domains of influence (disseminating science, creating awareness, catalyzing action, effecting change, and shaping the future) are not necessarily chronological and can be used in any order. The degree of impact is not necessarily a progression; therefore, events captured may not be reflected at every domain. The indicators listed under each domain are examples and may be used in creating your narrative. A program may tailor the use of these indicators, and others that are not listed, to its work.

 Top of Page

How is this framework different from the evaluation framework?

This framework is not a replacement for the evaluation framework but complementary to it. CDC Science is the process element of an “input-process-output” logic model, and the Science Impact elements measure output, outcome, and impact in the logic model.

Is the use of the Scientific Impact Framework mandatory?

No. It is not mandated; we hope that this creates an agency-wide culture change. Using this model will promote a culture change from quantifying citations of CDC science to using a combination of narrative and quantitative and qualitative data that capture what changes are occurring as a result of CDC science. It may prove to be useful for you and your program. The Science Impact Framework provides a way to track immediate or intermediary impact of your work, current projects (prospective) or completed (retrospective) projects.

 Top of Page

Will the framework be used consistently across CIOs?

The framework may be applied specifically to your CIOs work, which will differ across CIOs. OADS hopes to provide a training course through CDC University; this course will provide additional details and allow for interactive discussions. If you have further questions or if you would like a presentation at your Division or Center meetings, you may contact Dr. Mary Ari at MKB9@cdc.gov.

How can I use this tool for my program?

It does not matter whether we are talking about your results of research, guidelines and recommendations, new technology, new methods; no matter what we do, we inform the world about it in writing. So, dissemination of the science is a good place to begin tracing. What happened after the information was disseminated or released? Obviously, it is still all about following the paper trail, but the detail is in how this information is used. It is important to focus on how the knowledge is being used and not counting how often and by whom it is used only.

The Science Impact Framework may be used to support further project development, provide a narrative that may be used to respond to inquiries [e.g., the Government Accountability Office (GAO) and QPR], and communicate with the public and partners. The framework can also be used in funding opportunity announcements to provide a means of tracking impact of projects or research, assess trajectories, and shape the direction towards the ultimate goal. Generated narratives using this framework may be used in annual reports internally and externally.

 Top of Page

When should one use the framework?

The framework may be used to measure prospectively impact of an ongoing or new body of work (e.g., research, guidelines and recommendations, and technology) and retrospectively of completed and disseminated work.

Examples of prospective use of the framework include:

1. 1. 1. For a funding opportunity (FOA), the framework may provide recipients of grants a way of organizing their material as immediate measures of their projects. The final outcomes of these projects may take several years, and the framework provides immediate and future ways to measure a project’s impact.
      2. For tracking the use of a recently developed and disseminated Guideline and Recommendation

Examples of retrospective use of the framework include:

1. 1. 1. For a published manuscript, you may identify where it has been cited and the context in which it was cited.
      2. For a completed project, you may see how the multiple parts and individual components create an impact at various stages of the project.

 Top of Page

Is there training involved on how to apply the framework to my individual project?

OADS provided a training course through CDC University and featured the Science Impact Framework (SIF) in the January 2014 CDC Public Health Grand Rounds session.

What resources are available in helping me apply the framework to my project?

The Stephen B. Thacker CDC Library may assist you in coordinating bibliographies. OADS may provide advisement and consultation, and you may contact Dr. Mary Ari at MKB9@cdc.gov.

 Top of Page

To date, where has this framework been applied?

The framework has been applied to case studies selected from CDC Public Health Grand Rounds, Shepard award winning manuscripts, and MMWR articles to name a few. These case studies cover a broad set of activities across the agency, including basic research, laboratory, and epidemiology; guidelines and recommendation; surveillance; infectious diseases; non-communicable diseases; and meta-analyses and evaluations.

If I have applied this framework and created a narrative for my project or program and I feel like it may be helpful to others, where do I share my example?

OADS would like to hear from you – please contact Dr. Mary Ari at MKB9@cdc.gov.

If I have suggestions or comments, who do I contact?

OADS would like to hear from you – please contact Dr. Mary Ari at MKB9@cdc.gov.

 Top of Page