A Guide for Reporting the
Four Core
Measures Required of Drug
Support Program Grantees
March 21, 2008
By:
In Collaboration With
The Association for the Study and
A Guide for Reporting the
Four Core
Measures Required of Drug
Support Program Grantees
March 21, 2008
By:
In Collaboration With
The Association for the Study and
Section 1 – DFC Core measures Background Information
1.2
THE FOUR CORE MEASURES AND HOW DFC USES THEM
1.3 PROCEDURES
THAT DFC GRANTEES CAN USE TO COLLECT DATA ON THE FOUR CORE MEASURES
1.4 BEST
QUESTIONS TO MEET THE FOUR CORE MEASURES
1.5 ADDITIONAL INFORMATION ABOUT THE FOUR CORE MEASURES
1.6 REVIEW OF ALL SURVEY INSTRUMENTS
1.7 FREQUENTLY ASKED QUESTIONS
SECTION 2 – reviewed state and national/multi-state
surveys
2.1 COLLECTING AND REPORTING CORE MEASURES DATA
2.2
SURVEYS THAT ADDRESS THE CORE MEASURES
2.3
ASSISTANCE FROM STATE SURVEY MANAGERS
Purpose of the Guide
A
Guide for Reporting the Four Core Measures Required of Drug Free Communities (
How to Use the Guide
Grantees interested in learning about the four core measures and their purpose, how to best collect data on them, and what to do if they cannot find their survey on the pull-down menu of COMET should review Section 1. Grantees who are looking for specific guidance on how to report their core measure data in COMET should refer to Section 2. You can find the survey your coalition is using by either (1) looking through the list of reviewed surveys listed under your state, or (2) looking through the list of National/Multi-State Surveys that are conducted in more than one state in the U.S.
It is important that grantees understand that the questionnaires presented and described in Section 2 (Reviewed State and National/Multi-State Surveys) will continue to be updated as grantees notify us of additional questionnaires they are administering that are not included in the guide currently. We will add new questionnaires to subsequent editions of the Guide with conversion instructions and examples as appropriate. Section 1.7 provides instructions for submitting additional questionnaires for inclusion in the Guide .
Grantees looking for specific guidance on selecting or designing a survey that collects data on all four core measures can (1) refer to Section 1.4 of the Guide for suggested questions for each core measure and substance, (2) look for questionnaires in their state that collect data on all four core measures (i.e., those with an asterisk [*] by them in Section 2), or (3) contact the person(s) in their state who are responsible for administering youth substance abuse surveys (i.e., state survey managers). Contact information by state and talking points/questions for use with state survey managers are located in Section 2.
The Four Core
Measures of
The four core measures are the
indicators of community-level substance use that are required by the Government
Performance and Results Act (GPRA) for the DFC Support Program. The core measures are defined as:
The percentage of respondents who report using alcohol, tobacco, or marijuana at least ONCE in the past 30 days
The average age that respondents report first trying alcohol, tobacco, or marijuana
The percentage of respondents who report that regular use of alcohol, tobacco, or marijuana has moderate risk or great risk
Regular use is defined for alcohol as one or two drinks of an alcoholic beverage (beer, wine, liquor) nearly every day.
Regular use is defined for tobacco as one or more packs of cigarettes a day.
Regular use for marijuana is not defined.
The percentage of respondents who report their parents feel regular use of alcohol is wrong or very wrong
The percentage of respondents who
report their parents feel ANY use of
cigarettes or marijuana is wrong or very wrong
DFC grantees demonstrated their ability to collect and report on the four core measures in their applications and are expected to provide data for these measures to the Substance Abuse and Mental Health Services Administration (SAMHSA) at a minimum of every two years. The grantees are requested to report data by school grade and gender, the preferred population is school-aged youth, grades 6 to 12. All of the questionnaires outlined in this guide ask respondents their grade and gender.
How Do
Key
Data collection requires identifying from whom you will collect data (i.e., sample) and procedures for gathering the data. This section contains some helpful information grantees can use to identify their sample and procedures for gathering data.
Sample
In general,
Procedures
Questionnaires, like those described in this guide, typically
are used to collect data from large populations such as the students living in
the target areas served by
The best questions on the four core measures reflect the conceptual definitions of the measures as defined in Section 1.2. Questionnaires contain a mixture of questions that (1) exactly reflect the conceptual definitions of the core measures, (2) closely reflect the core measures (after a simple mathematical conversion), or (3) are worded such that they do not reflect the core measures. When a survey does not collect data on the required core measures or when data cannot be mathematically converted for use, grantees can supplement their existing instrument with surveys or questions that meet the core measures.
Detailed descriptions of each of the most commonly used questionnaires are presented in Section 2. The descriptions indicated whether a questionnaire collects data on the required four core measures and identifies the particulare questions that ask about each core measure. For questionnaires in your state that collect data on all four core measures, review the list of surveys and identify questionnaires with an asterisk [*] by them. If none of your state surveys collect data on all four core measures, your coalition can use the questions below to supplement your existing survey. These questions are phrased in the exact language required when reporting the four core measures:
Alcohol
Question: On how many occasions (if any) have you had alcoholic beverages (beer, wine, or hard liquor) to drink – more than just a few sips – during the past 30 days?
Response options:
0 occasions
1-2 occasions
3-5 occasions
6-9 occasions
10-19 occasions
20-39 occasions
40 or more occasions
Tobacco
Question: How frequently have you smoked cigarettes during the past 30 days?
Response options:
Not at all
Less than one cigarette per day
One to five cigarettes per day
About one-half pack per day
About one pack per day
About one and one-half packs per day
Two packs or more per day
Marijuana
Question: On how many occasions (if any) have you used marijuana during the past 30 days?
Response options:
0 occasions
1-2 occasions
3-5 occasions
6-9 occasions
10-19 occasions
20-39 occasions
40 or more occasions
Alcohol
Question: How old were you when you first had more than one sip or two of beer, wine, or hard liquor (for example, vodka, whiskey, or gin)?
Response options:
Never have
10 or younger
One option each for ages 11-16
17 or older
Tobacco
Question: How old were you when you first smoked a cigarette, even just a puff?
Response options:
Never have
10 or younger
One option each for ages 11-16
17 or older
Marijuana
Question: How old were you when you first smoked marijuana?
Response options:
Never have
10 or younger
One option each for ages 11-16
17 or older
Alcohol
Question: How much do you think people risk harming themselves (physically or in other ways) if they take one or two drinks of an alcoholic beverage (beer, wine, liquor) nearly every day?
Response options:
No risk
Slight risk
Moderate risk
Great risk
Tobacco
Question: How much do you think people risk harming themselves (physically or in other ways) if they smoke one or more packs of cigarettes per day?
Response options:
No risk
Slight risk
Moderate risk
Great risk
Marijuana
Question: How much do you think people risk harming themselves (physically or in other ways) if they smoke marijuana regularly?
Response options:
No risk
Slight risk
Moderate risk
Great risk
Alcohol
Question: How wrong do your parents feel it would be for you to drink beer, wine, or hard liquor (for example, vodka, whiskey, or gin) regularly?
Response options:
Very wrong
Wrong
A little bit wrong
Not at all wrong
Tobacco
Question: How wrong do your parents feel it would be for you to smoke cigarettes?
Response options:
Very wrong
Wrong
A little bit wrong
Not at all wrong
Marijuana
Question: How wrong do your parents feel it would be for you to smoke marijuana?
Response options:
Very wrong
Wrong
A little bit wrong
Not at all wrong
Section 2 also provides instructions and examples on how to calculate the data for entry into the COMET system. If you have questions about the instructions provided, please contact a national evaluation team member by email at survey@capablecommunity.com. If you have questions about how to use the COMET system, contact COMET Technical Support by phone at (240) 223–3002 or toll free at (877) 654–6740 or contact them by email at COMEThelp@cdmgroup.com.
Several on-line resources provide more in-depth information
about the four core measures required of
i. Click on “Monitoring the Future” button
ii. Click on “View Instrument” button
Planning and Using Survey Research Projects: A Guide for Grantees of the Robert Wood Johnson Foundation (RWJF) is a guide that can help grantees:
Although the guide was designed for RWJF grantees, the material is very relevant to anyone conducting prevention practice research. This guide is available at: http://www.rwjf.org/files/publications/RWJF_SurveyGuide_0804.pdf.
ONDCP recommends that grantees use a survey that incorporates questions that adequately reflect the core measures definitions. To identify which surveys collect data that reflect the core measures definitions, it is important for Battelle’s evaluation team to review all survey questions being used by grantees to collect core measures outcome data. As a result, the COMET drop-down menu for source of core measures outcome data has been changed. The new drop-down menu consists of only those questionnaires that Battelle has reviewed using the core measures guidelines (i.e., they consistently collect information on one or more of the core measures). Quality assurance will lead to greater confidence in the successes we find among DFC coalitions.
If the questionnaire you are using has not been reviewed and added to the drop-down menu, please request that Battelle review your questionnaire by emailing a copy of the questionnaire to your Program Officer and to the Battelle evaluation team at survey@capablecommunity.com. To expedite the review process, please include your coalition name and contact information, grant SP number, Program Officer name, and survey name in the email. Surveys and the supporting information also can be faxed to (301) 519-0724. Once your questionnaire is reviewed, it will be added to the COMET drop-down menu for your use in reporting data. New state and national/multi-state surveys also will be added to Section 2 of the Guide during its next update. Because the review process takes time, it is important that you leave yourself enough time to have your questionnaire reviewed as you are still required to submit your core measure data on time. Please remember that the purpose of this review is to determine if your outcome data could be included as part of the national evaluation of the DFC Support Program and not to determine your compliance with grant requirements.
In the core measures section of COMET you will find boxes asking you to report
“overall sample size” data for Age of Onset, and “sample size” data for the
other three core measures. The “overall sample size” is the total number of
people that responded to the questions about Age of Onset. This includes
responses that are not used to calculate the average age of onset (i.e., youth
that have never used alcohol, tobacco, and marijuana). The “sample size” is the
number of people who answered the question and whose responses were used to
determine the percentage for Past 30-Day Use, Perception of Risk, and
Perception of Parental Disapproval. It is important to note the difference
between “overall sample size” and “sample size” because the information
is essential for enabling the data across coalitions to
be combined and analyzed in order to determine how
Yes. These outcome data are described in the grant application and coalitions are expected to collect the information.
What do I do if I have data from multiple schools, multiple counties, etc.?
If the
data are from the same source (e.g., YRBS, Adolescent ADAS, ADAS–CM, etc.),
combine them to cover the area that best represents your coalition’s target
area. Please indicate under the core measures questions in COMET the source of
the data and whether the participants in the combined area represent an area
that is larger (i.e., covers your target area and additional areas), smaller
(i.e., covers only a portion of the target area), or the same as the target
area. Use the pull-down menu corresponding to the phrase, “Compared
to the Target Area, the Geographical Area covered by this data is,” to indicate
smaller, larger, or the same.
If the sample sizes of your multiple data sources are the same, calculate the average percent by adding the average from each of your data sources (e.g., three schools with 35, 20, 32 percent; 35+20+32 = 87), then divide by the number of data sources (e.g., 87 divided by 3 = 29). The average percent in this example is 29 percent. Then provide the total sample size (n) by adding the number of people participating in each survey and reporting the sum in the appropriate row.
If the sample sizes of your multiple data sources are different, you will need to convert your percentages to number of people (n), sum the number of people, and divide by the total sample size.
Example: Past 30-Day Use of Alcohol in Three Counties
County 1, 25% of the youth have used alcohol in the past 30-days and the sample size (n) = 800 (.25 x 800 = 200)
County 2, 20% of the youth have used alcohol in the past 30-days and the sample size (n) = 200 (.20 x 200 = 40)
County 3, 35% of the youth have used alcohol in the past 30-days and the sample size (n) = 600 (.35 x 600 = 210)
The total number of youth that have used alcohol in the past 30-days is 450 and the total sample size (n) = 1,600. The percentage of youth that used alcohol in the past 30-days across the three counties is 28.13% (450 divided by 1,600)
For the average age of onset, you will need to find the average across all sources, which is a bit more work. For each source, you will need to take the average age and multiply it by the sample size (i.e., the number of people completing the question), which we will call the “sum.” Add all the sums for that population and outcome and then divide by the total sample size (n) for all the samples (e.g., schools or school districts).
Example: Three schools
School 1, average age is 15.6 and the sample size (n) = 800 (15.6 x 800 = 12,480)
School 2, average age is 16.2 and the sample size (n) = 650 (16.2 x 650 = 10,530)
School 3, average age is 16 and the sample size (n) = 745 (16 x 745 = 11,920)
The total of the sums is 34,930 and the total sample size (n) = 2,195. The average age for the three schools is 15.9 (34,930 divided by 2,195).
For help with combining data
sources (i.e., questionnaires), contact COMET Technical Support by phone at (240)
223–3002 or toll free at (877) 654–6740 or contact them by email at COMEThelp@cdmgroup.com.
That information is not useable and should not be reported. The requirement for reporting this data will not have been met.
There is nothing further to do unless required by your project officer.
For each grade that was combined, please enter the same percent or average age under each grade for that outcome. For the sample size (n), divide by the number of grades and put that number in for each of the grades that were part of the combined data.
Example: Combined grades
Combined past 30-day use of marijuana for grades 6 to 8 and 10 to 12 is 12 percent, and the sample size (n) is 300. List “12” under the “%” row for Grades 6, 7, 8, 10, 11, and 12. Enter a sample size (n) of 50 for each grade (300 divided by 6).
Yes. The combined data will provide a complete, updated data set for analysis. These data represent the most complete and recent information on youth for this fiscal year. In the appropriate section of COMET, please indicate that this is a new data report. Indicate that the area covered was less than your target area, and when asked the method, indicate under “other method” that you collected the data from a certain number of faith-based organizations. However, please remember that you are expected to collect outcome data for your entire target community.
Is there an assumption that the questionnaires used by coalitions are
all school-based surveys?
No. There is an assumption, however, that youth who responded to the questions can be assigned to an accurate and appropriate school grade for reporting purposes.
No. Please enter the most recent data that you have for your entire target area as instructed in COMET. No additional files should be sent.
When a survey does not collect data on the required core measures or when data cannot be mathematically converted for use, grantees can supplement their existing instrument with surveys or questions that meet the core measures. For questionnaires in your state that collect data on all four core measures, review the list of surveys under your state (Section 2) and identify questionnaires with an asterisk [*] by them. If none of your state surveys collect data on all required four core measures, your coalition can use the list of survey questions in Section 1.4 of the Guide to supplement your existing survey. These questions are phrased in the exact language required when reporting the four core measures.
“Never used” or “never have” data is not reported in COMET. However, knowing the number of youth who never used is useful in determining the percentage of people who used as well as the overall sample size, both of which are entered into COMET.
If the questionnaire you are using is not on the current drop-down menu, please request that Battelle review your questionnaire by emailing a copy of the questionnaire to your Program Officer and to the Battelle evaluation team at survey@capablecommunity.com. To expedite the review process, include your contact information, grant SP number, Project Officer name, and survey name in the email. Surveys and the supporting information also can be faxed to (301) 519-0724. Once your questionnaire is reviewed, it will be added to the COMET drop-down menu for your use in reporting data. Because the review process takes time, it is important to leave yourself enough time to have your questionnaire reviewed as you are still required to submit your core measure data on time. The purpose of this review is to determine if your outcome data could be included as part of the national evaluation of the DFC Support Program and not to determine your compliance with grant requirements.
To develop the instructions for how to report data in the core measures section of COMET, the national evaluation team reviewed the survey questionnaires to determine if the data they collect can be included as part of the national evaluation of the DFC Support Program (i.e., meets national evaluation requirements). National evaluation results are very important in demonstrating the impact of the DFC program on substance abuse outcomes. The ability or inability to include data you collect on the four core measures depends on the relationship of your data elements to the conceptual definitions of the core measures as defined in this guide.
The following state and national/multi-state surveys meet the requirements for some or all of the core measures. Detailed descriptions of each of these questionnaires are available by clicking on the survey name. The descriptions (1) identify which survey questions ask about each core measure, and (2) provide instructions and examples for how to calculate the data for reporting in COMET. Surveys that meet the evaluation requirements for all core measures and substances are denoted by an asterisk [*].
National or Multi-Community
Surveys
State Surveys
Healthy
Kids
Healthy
Kids
Colorado Youth Risk Behavior Survey (2007)
Connecticut
Governor's Prevention Initiative for Youth: Youth Survey (Grades 6-12)
<
