The purpose of the Canadian Community Health Survey - Mental Health (CCHS - Mental Health) is to collect information about mental health status, access to and perceived need for formal and informal services and supports, functioning and disability, and covariates.
Data release – September 18, 2013
The survey will provide a comprehensive look at mental health with respect to who is affected by selected mental disorders as well as positive mental health. It will also examine access to and utilization of formal and informal mental health care services and supports. It will look at how people are functioning regardless of whether they have a mental health problem.
The objectives of the Canadian Community Health Survey - Mental Health are:
(1) To assess the mental health status of Canadians on both illness and positive mental health continuums through selected mental and substance disorders, mental health problems, and well-being;
(2) To assess timely, adequate, and appropriate access to and utilization of formal and informal mental health services and supports as well as perceived needs;
(3) To assess functioning, ability and disability in relation to mental health and illness;
(4) To examine links between mental health and social, demographic, geographic, and economic variables or characteristics (covariates); and
(5) To evaluate changes in patterns of mental health, service use, and functioning from the 2002 CCHS on Mental Health and Well-being.
The data collected from the survey will be used by Statistics Canada, Health Canada, the Public Health Agency of Canada, federal and provincial departments, the Mental Health Commission of Canada, as well as universities, pharmaceutical companies, and mental health services and support providers to fill data gaps in understanding mental health. Policy makers and researchers will use this information to develop policies and programs that properly meet the mental health needs of Canada's population. The media will use this information and thus help raise general awareness about health, an issue of concern to all.
The Canadian Community Health Survey - Mental Health covers the population 15 years of age and over living in the ten provinces. Excluded from the survey's coverage are: persons living on reserves and other Aboriginal settlements; full-time members of the Canadian Forces and the institutionalized population. Altogether, these exclusions represent about 3% of the target population.
The Canadian Community Health Survey - Mental Health (CCHS - Mental Health) questionnaire was developed by Statistics Canada in collaboration with stakeholders from Health Canada and the Public Health Agency of Canada, the Provincial Health Ministries, an expert advisory group consisting of specialists from Health Canada, the Public Health Agency of Canada, the Mental Health Commission of Canada, and academic experts. Content was chosen using the following set of criteria:
- Issues identified as data gaps from the stakeholder consultations
- Significant number of people affected by the targeted issue
- Significant impact on family, community, and health care costs
- Data has potential for health improvement with policy intervention
- Issues identified as priority for the support/development of programs and policy, surveillance requirements and/or research
- Comparability with previous cycle of CCHS Mental Health survey
The CCHS - Mental Health questions are designed for computer-assisted interviewing (CAI), meaning that, as the questions were developed, the associated logical flow into and out of the questions was programmed. This includes specifying the type of answer required, the minimum and maximum values, on-line edits associated with the question and what to do in case of item non-response.
In collaboration with Statistics Canada's Questionnaire Design Resource Centre, the questionnaire was subjected to two phases of qualitative testing, which took place in March and July of 2010 and consisted of one-on-one interviews. The objective was to evaluate respondent reactions to and understanding of the survey, as well as their willingness to respond to the questions. The qualitative testing was also used to obtain estimates for the various sections' time.
A pilot survey was conducted in April 2011. The pilot survey for the Canadian Community Health Survey on Mental Health tested survey content, methodology, computer applications, interviewer procedures, reference material and data processing techniques in preparation for the main survey collection.
This is a sample survey with a cross-sectional design.
Sample Size and Allocation
To provide reliable estimates at the provincial level, and given the budget allocated to the survey, a sample of 27,500 respondents was desired. The goal was to produce reliable estimates by province for four age groups (15-24, 25-44, 45-64 and 65+) and by sex.
Because provinces vary greatly in population size and reliable estimates were required both at national and provincial levels, a two-step strategy was used to allocate the sample to the provinces. First, 125 sample units were allocated to each domain of interest (8 age/sex groups) in each province. The remaining 17,500 units were allocated to the provinces using a power allocation method with power q=0.7, based on the estimated population aged 15 and over in each province. The total sample size of any given province was found by adding the sizes obtained in the two steps.
Sample sizes were enlarged before data collection to take into account out-of-scope and vacant dwellings and anticipated non-response. A total raw sample of 43,030 dwelling was therefore selected.
A three-stage design was used to select the sample of respondents for the 2012 CCHS - Mental Health. First, geographical areas called clusters were selected. Households were then selected within each sampled cluster and, finally, one respondent per household was randomly selected.
The CCHS - Mental Health used the area frame designed for the Labour Force Survey (LFS) as its area frame. Thus, the sampling plan of the LFS had to be considered in selecting the CCHS dwelling sample. The LFS plan is a complex two stage stratified design in which each stratum is formed of clusters. The LFS first selects clusters using a sampling method with a probability proportional to size (PPS). In the second stage, dwelling lists are prepared for each cluster and a systematic sample of dwellings, or households, is selected from these lists.
Once the dwelling sample had been chosen, the next step was to select a member in each household. Upon visiting a selected dwelling, the household composition was obtained. One respondent was then selected at random among all eligible respondents using various selection probabilities based on age and household composition.
Data collection for this reference period: 2012-01-02 – 2012-12-31
Responding to this survey is voluntary.
Data are collected directly from survey respondents.
Collection for the Canadian Community Health Survey - Mental Health took place between January 2012 and December 2012. Over the collection period, a total of 25,113 valid interviews were conducted using a type of computer-assisted interviewing (CAI) called computer assisted personal interviewing (CAPI).
Computer-assisted interviewing (CAI) offers two main advantages over other collection methods. First, computer-assisted interviewing offers case management system and data transmission functionality. Second, computer-assisted interviewing allows for custom interviews for every respondent based on their individual characteristics and survey responses.
Before the start of each collection period, introductory letters and brochures explaining the purpose of the survey were sent to the sampled households. These explained the importance of the survey and provided examples of how the CCHS - Mental Health data would be used.
Units selected from the area frame were interviewed by decentralized field interviewers using CAPI. CAPI interviewers were trained to make an initial personal contact with each sampled dwelling. Most interviews (87%) were conducted in person.
Interviewers were instructed to make all reasonable attempts to obtain interviews. Numerous repeat visits to the dwelling were made at different times on different days. In May 2012, to facilitate interviewing efficiency and to improve response rate, interviewers were authorized to make initial contact by telephone when possible in order to collect household information, select the respondent, and set an appointment for an interview. If the respondent was available at that time, they were offered the option to complete the interview immediately over the phone. No proxy interviews were permitted for this survey.
For individuals who at first refused to participate in the survey, a letter was sent from the nearest Statistics Canada Regional Office to the respondent, stressing the importance of the survey and the household's collaboration. This was followed by a second visit (or call) from a senior interviewer, a project supervisor or another interviewer to try to convince respondent of the importance of participating in the survey.
The CCHS - Mental Health sample was divided into six non-overlapping two-month collection periods. Regional collection offices were instructed to use the first 4 weeks of each collection period to resolve the majority of the sample, with the next 4 weeks being used to finalise the remaining sample and to follow up on outstanding non-response cases. Cases initially coded to "No contact" or "Absent for duration of survey" were resent in an additional collection period from November 1 to December 30, 2012.
Most editing of the data was performed at the time of the interview by the computer-assisted personal interviewing (CAPI) application. It was not possible for interviewers to enter out-of-range values and flow errors were controlled through programmed skip patterns. For example, the CAPI application ensured that questions that did not apply to the respondent were not asked.
In response to some types of inconsistent or unusual reporting, warning messages were invoked but no corrective action was taken at the time of the interview. Where appropriate, edits were instead developed to be performed after data collection at Head Office. Inconsistencies were usually corrected by setting one or both of the variables in question to "not stated".
Pre-coded answer categories were supplied for all suitable variables. Interviewers were trained to assign the respondent's answers to the appropriate category.
In the event that a respondent's answer could not be easily assigned to an existing category, several questions also allowed the interviewer to enter a long-answer text in the "Other-specify" category. Following collection, all such questions were reviewed in head office processing. For some of these questions, write-in responses were coded into one of the existing listed categories if the write-in information duplicated a listed category.
Finally, selected health indicators underwent a validation process after final microdata files are produced. Estimates for all geography levels by sex and by age groups were compared to estimates from the annual CCHS and/or the 2002 CCHS - Mental Health and Well-being. This process allows confirmation that estimates of key indicators are acceptable.
The household income variable was imputed for the CCHS-Mental Health. Missing values due to either respondent refusal or respondent's lack of knowledge of household income were completed using statistical techniques. The main variable of interest was INC_3: 'Total household income - best estimate' but all variables that were derived based on income were also affected. The income variables along with an imputation flag (INCFIMP4) indicating which values were imputed are provided on the data file.
The personal income of some respondents was also imputed to correct for a problem with the CAPI application. People who didn't live alone and reported no source of personal income were not asked for their personal income, as should have been the case. Based on patterns observed from the annual CCHS and on their household income, a personal income was assigned to these persons, 95% of them receiving an income of zero. An imputation flag (INCFIMP5) indicating which values were imputed is provided on the data file.
In order for estimates produced from survey data to be representative of the covered population and not just the sample itself, users must incorporate the survey weights into their calculations. A survey weight is given to each person included in the final sample, that is, the sample of persons having responded to the survey. This weight corresponds to the number of persons in the entire population that the respondent represents. Seven separate adjustments are part of the weighting strategy.
1. Initial weight
2. Removal of out-of-scope units
3. Household non-response
4. Creation of person level weight
5. Person non-response
The final CCHS - Mental Health weight can be found on the data file with the variable name WTS_M.
In order to determine the quality of an estimate, the variance must be calculated. Because the CCHS - Mental Health uses a multi-stage survey design, there is no simple formula that can be used to calculate variance estimates. Therefore, an approximation method is needed. Coefficient of variation, standard deviation and confidence intervals can then be calculated from the variance. The bootstrap re-sampling method used in the survey involves the selection of simple random samples known as replicates, and the calculation of the variation between the estimates from replicate to replicate. In each stratum, a simple random sample of (n-1) of the n clusters is selected with replacement to form a replicate. Note that since the selection is with replacement, a cluster may be chosen more than once. In each replicate, the survey weight for each record in the (n-1) selected clusters is recalculated using the same steps as for the main weight to obtain the final bootstrap weights. The entire process (selecting simple random samples and recalculating the weights) is repeated 500 times to produce 500 bootstrap weights for each respondent. To obtain the bootstrap variance estimator, the point estimate for each of the 500 samples must be calculated. The standard deviation of these estimates is the bootstrap variance estimator.
A macro program called Bootvar was developed by Statistics Canada to give users easy access to the bootstrap method. The Bootvar program is available in SAS and SPSS formats, and is made up of macros that calculate the variances of totals, ratios and differences between ratios, as well as linear and logistic regressions.
The bootstrap weights are considered confidential and thus, are not available on the Public Use Microdata File (PUMF). In order to supply coefficients of variation that will be applicable to a wide variety of categorical estimates produced from a PUMF and that can be readily accessed by the user, a set of Approximate Sampling Variability Tables is produced. These "look-up" tables allow the user to obtain an approximate coefficient of variation based on the size of the estimate calculated from the survey data. All coefficients of variation in the Approximate Sampling Variability Tables are approximate and, therefore, unofficial.
To ensure the survey met its objectives (see "Survey Description"), the CCHS - Mental Health content was developed based on a multi-stage consultation process. A Steering Committee and an Advisory Board comprised of authorities from the provincial and territorial Ministries of Health, the Canadian Institute for Health Information, Health Canada, the Public Health Agency of Canada as well as an Expert Advisory group with representatives from Health Canada, the Public Health Agency of Canada, the Mental Health Commission of Canada, Mood Disorders Society of Canada, and academic experts determined the concepts and focus. Experts and stakeholders advised on the selection of content and provided recommendations on appropriate and proven collection instruments and indices. The resulting data are recognized as valid measures of concepts such as depression, bipolar disorder, generalized anxiety disorder, alcohol and drug abuse and dependence, positive mental health, childhood maltreatment, and perceived need for mental health care services.
Throughout the collection process, control and monitoring measures were put in place and corrective action was taken to minimize non-sampling errors. These measures included response rate evaluation, monitoring of non-item non-response, and reported, non-reported data evaluation and on site observation of interviews.
Once processing steps were completed, three data validation steps were undertaken. First, a validation program was run in order to compare estimates for the health indicators taken from the common content with the 2002 Canadian Community Health Survey - Mental Health, the annual Canadian Community Health Survey, and the General Social Survey. This validation was performed by age group, sex, and province. Significant differences are examined further to find any anomalies in data.
Also, the work of analysts who use the CCHS data to publish analytical articles on specific themes allows for an in-depth look at many variables of the survey and represents a very effective way to find error.
Last, an external validation step was also part of the validation process. Share files were sent before release to provincial and federal partners for a two-week examination period. They could then scrutinize the data and inform Statistics Canada of any concerns or anomalies related to data quality.
Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.
The public use microdata files (PUMF) are developed from the master files using a technique that balances the need to ensure respondent confidentiality with the need to produce the most useful data possible at the provincial or regional level. The PUMF must meet stringent security and confidentiality standards required by the Statistics Act before they are released for public access. To ensure that these standards have been achieved, each PUMF goes through a formal review and approval process by an executive committee of Statistics Canada. Variables most likely to lead to identification of an individual are deleted from the data file or are collapsed to broader categories. Some values of indirect identifiers (mostly socio-demographic variables) were suppressed.
Users requiring access to information excluded from the microdata files may purchase custom tabulations, or access the master files through the Research Data Centres program or the Remote Access program. Outputs are vetted for confidentiality before being given to users.
Estimates in the main body of a statistical table are rounded to the nearest hundred units using the normal rounding technique. If the first or only digit dropped is zero to four, the last digit retained is not changed. If the first or only digit dropped is five to nine, the last digit retained is raised by one. Marginal sub-totals and totals in statistical tables are derived from their corresponding unrounded components and then are rounded themselves to the nearest 100 units using normal rounding methods. Averages, proportions, rates and percentages are computed from unrounded components (for example, numerators and/or denominators) and then are rounded themselves to one decimal using normal rounding. In normal rounding to a single digit, if the final or only digit dropped is zero to four, the last digit retained is not changed. If the first or only digit dropped is five to nine, the last digit retained is increased by one. Sums and differences of aggregates (or ratios) are derived from their corresponding unrounded components and then are rounded themselves to the nearest 100 units (or the nearest one decimal) using normal rounding. Under no circumstances are unrounded estimates, published or otherwise, released. Unrounded estimates imply greater precision than actually exists.
In total, 36,443 of the selected units in the CCHS - Mental Health were in-scope for the survey. Out of these, 29,088 households agreed to participate in the survey, resulting in an overall household-level response rate of 79.8%. Among these responding households, 29,088 individuals (one per household) were selected to participate in the survey, out of which a response was obtained for 25,113 individuals, resulting in an overall person-level response rate of 86.3%. At the Canada level, this yields a combined (household and person) response rate of 68.9% for the CCHS - Mental Health. Table 8.1 provides response rates by province and age group.
Since it is an unavoidable fact that estimates from a sample survey are subject to sampling error, sound statistical practice calls for researchers to provide users with some indication of the magnitude of this sampling error. The basis for measuring the potential size of sampling errors is the standard deviation of the estimates derived from survey results. However, due to the large variety of estimates that can be produced from a survey, the standard deviation of an estimate is usually expressed relative to the estimate to which it pertains. This resulting measure, known as the coefficient of variation (CV) of an estimate, is obtained by dividing the standard deviation of the estimate by the estimate itself and is expressed as a percentage of the estimate.
Statistics Canada commonly uses CV results when analyzing data and urges users producing estimates from the CCHS - Mental Health data files to do so as well.
Over a large number of observations, randomly occurring errors will have little effect on estimates derived from the survey. However, errors occurring systematically will contribute to biases in the survey estimates. Considerable time and effort was devoted to reducing non-sampling errors in the CCHS - Mental Health. Quality assurance measures were implemented at each step of data collection and processing to monitor the quality of the data.
A major source of non-sampling errors in surveys is the effect of non-response on survey results. The extent of non-response varies from partial non-response (failure to answer one or a few questions) to total non-response. Partial non-response to the CCHS - Mental Health was minimal; once the questionnaire was started, it tended to be completed with very little non-response. Total non-response occurred either because a person refused to participate in the survey or because the interviewer was unable to contact the selected person. Total non-response was handled by adjusting the weight of persons who responded to the survey to compensate for those who did not respond. Non-response to any particular question (item non-response) is generally low, but may be higher in some modules for various reasons.
For more details on data accuracy measures and response rates, please refer to the User Guide documents. To obtain a copy of this documentation contact Client Services ((613-951-1746; fax: 613-951-0792; email@example.com).
- CCHS - Mental Health User Guide
- CCHS - Mental Health Derived Variable (DV) Specifications
- CCHS - Mental Health Alphabetic Index
- CCHS - Mental Health Topical Index
- CCHS - Mental Health Record Layout