Canadian Community Health Survey - Mental Health and Well-being (CCHS)

Detailed information for 2002 (Cycle 1.2)

The major objectives are to provide national estimates of major mental disorders and problems, and to illuminate the issues associated with disabilities and the need and provision of health care.

Data release - September 3, 2003

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy
• Documentation

Description

Cycle 1.2 mainly measures aspects linked to the mental health of Canadians. This cycle was then named "Canadian Community Health Survey-Mental Health and Well-being". The primary objectives of the CCHS Mental Health and Well-being are to:

- Provide timely, reliable, cross-sectional estimates of mental health determinants, mental health status and mental health system utilization across Canada;
- Determine prevalence rates of selected mental disorders to assess the impact of burden of illness;
- Juxtapose access and utilization of mental health services with respect to perceived needs; and
- Assess the disabilities associated with mental health problems to individuals and society.

As a key component of the Population Health Surveys Program of Statistics Canada, the CCHS helps fulfil broader requirements of health issues in Canada. These are:

- Aid in the development of public policy;
- Provide data for analytic studies that will assist in understanding the determinants of health;
- Collect data on the economic, social, demographic, occupational and environmental correlates of health;
- Increase the understanding of the relationship between health status and health care utilization.

In Canada, the primary use of the data is for health surveillance, such as in prevalence of disease and other forms of health research. The data are used extensively by the research community and other health professionals. Federal and provincial departments of health and human resources, social service agencies, and other types of government agencies use the information collected from the respondents to plan, implement and evaluate programs to improve health and the efficiency of health services. Non-profit health organizations and researchers in the academic fields use the information to make research to improve health. The media uses the results from the surveys to raise awareness about health, an issue of concern to all.

Statistical activity

This survey is a component of the Health Statistics Program.

• Additional documentation

Reference period: Varies according to the question (for example: "over the last 12 months", "over the last 6 months", "during the last week", etc.)

Subjects

• Health
• Health care services
• Lifestyle and social conditions
• Mental health and well-being
• Prevention and detection of disease

Data sources and methodology

Target population

The Mental Health Survey covers the population 15 years of age and over living in the ten provinces. Excluded from the survey's coverage are: persons living on reserves and other Aboriginal settlements; full-time members of the Canadian Forces and the institutionalized population. Altogether, these exclusions represent about 2% of the target population.

Instrument design

The content for this study is partly based on a selection of mental disorders from the World Mental Health Survey (WMH2000). The other content areas come from existing sources such as the Canadian Community Health Survey (Cycle 1.1) and other special studies.

An Expert Group of mental health professionals guided the content development and strategic direction of the study. As well, the survey program is supported by a standing Advisory Committee, comprised of provincial and territorial ministries of health, Health Canada and Canadian Institute on Health Information (CIHI) representatives. Finally, Statistics Canada has been working closely with the World Health Organisation (WHO) so that the final outcome will have international comparability.

The CCHS questions were designed for computer-assisted interviewing (CAI), meaning that, as the questions were developed, the associated logical flow into and out of the questions was programmed. This included specifying the type of answer required, the minimum and maximum values, on-line edits associated with the question and what to do in case of item non-response.

With CAI, the interview can be controlled based on answers provided by the respondent. On-screen prompts are shown when an invalid entry is recorded and thus immediate feedback is given to the respondent and/or the interviewer to correct inconsistencies. Another enhancement is the automatic insertion of reference periods based on current dates. Pre-filling of text or data based on information gathered during the interview allows the interviewer to proceed without having to search back for previous answers. This type of pre-fill includes such things as using the correct name or sex within the questions themselves. Allowable ranges/answers based on data collected during the interview can also be programmed. In other words, the questionnaire can be customized to the respondent based on data collected at that time or during a previous interview.

Qualitative testing for Cycle 1.2 took place in July and August, 2001 to evaluate respondent reactions with regards to the sensitivity of the subject matter and their ability to understand and willingness to respond to the questions.

A pilot test was conducted in February 2002 in the provinces of Saskatchewan and Quebec. The objectives of the pilot test were to determine the effectiveness of the training of interviewers and communication strategy and test the training procedures and material, to provide a preliminary indication of the response rates, and to test the computerized questionnaire.

• Additional documentation

Sampling

This is a sample survey with a cross-sectional design.

To provide reliable estimates at the provincial level, and given the budget allocated to the Cycle 1.2 component, a sample of 30,000 respondents was desired. Because provinces vary greatly in population size and reliable estimates are required both at national and provincial levels, the sample was allocated among provinces proportionally to the square root of the estimated population in each province.

Prior to the start of the data collection, the provinces of Ontario and Nova Scotia provided extra funds so that a larger sample of dwellings could be selected. The purpose of those buy-ins were to get sufficient sample size in order to provide reliable estimates for sub-provincial areas of geography.

Cycle 1.2 used the area frame designed for the Canadian Labour Force Survey (LFS) as its frame. The sampling plan of the LFS is a multistage stratified cluster design in which the dwelling is the final sampling unit. In the first stage, homogeneous strata were formed and independent samples of clusters were drawn from each stratum. In the second stage, dwelling lists were prepared for each cluster and dwellings, or households, were selected from the lists.

For the purpose of the plan, each province is divided into three types of regions: major urban centres, cities and rural regions. Geographic or socio-economic strata are created within each major urban centre. Within the strata, between 150 and 250 dwellings are regrouped to create clusters. Some urban centres have separate strata for apartments or for census enumeration areas (EA) in which the average household income is high. In each stratum, six clusters or residential buildings (sometimes 12 or 18 apartments) are chosen by a random sampling method with a probability proportional to size (PPS), the size of which corresponds to the number of households. The number six was used throughout the sample design to allow a one-sixth rotation of the sample every month for the LFS.

The other cities and rural regions of each province are stratified first on a geographical basis, then according to socio-economic characteristics. In the majority of strata, six clusters (usually census EAs) are selected using the PPS method. Where there is low population density, a three-step plan is used whereby two or three primary sampling units (PSU), which normally correspond to groups of EAs, are selected and divided into clusters, six of which are sampled. The selection is made at each step using the PPS method. Once the new clusters are listed, the sample is obtained using a systematic sampling of dwellings.

Sampling of respondents

Selection of individual respondents was designed to ensure adequate representation of young persons (15 to 24) and seniors (65 or older). The selection strategy was designed to consider user needs, cost, design efficiency, response burden and operational constraints1 . One person aged 15 or older was randomly selected from the sampled households. The probability of selection for each person in a household was defined as a function of the household composition.

Data sources

Data collection for this reference period: 2002-05-01 to 2002-12-31

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

The Cycle 1.2 questionnaire was administered using computer-assisted interviewing (CAI). Sample units selected from the area frame were interviewed using the Computer-Assisted Personal Interviewing (CAPI) method.

CAI offers a number of data quality advantages over other collection methods. First, question text, including reference periods and pronouns, is customised automatically based on factors such as the age and sex of the respondent, the date of the interview and answers to previous questions.

Second, edits to check for inconsistent answers or out-of-range responses are applied automatically and on-screen prompts are shown when an invalid entry is recorded. Immediate feedback is given to the respondent and the interviewer is able to correct any inconsistencies.

Third, questions that are not applicable to the respondent are skipped automatically

View the Questionnaire(s) and reporting guide(s) .

Error detection

Most editing of the data was performed at the time of the interview by the computer-assisted interviewing (CAI) application. It was not possible for interviewers to enter out-of-range values and flow errors were controlled through programmed skip patterns. For example, CAI ensured that questions that did not apply to the respondent were not asked.

In response to some types of inconsistent or unusual reporting, warning messages were invoked but no corrective action was taken at the time of the interview. Where appropriate, edits were instead developed to be performed after data collection at Head Office. Inconsistencies were usually corrected by setting one or both of the variables in question to "not stated".

Imputation

Due to some technical problems in certain skip patterns of the suicide module, some respondents were not asked the questions required for the calculation of the derived variables "12-month suicidal thought" and "12-month suicidal attempt". Consequently, important information was missing for those individuals (this represented around 5% of all respondents for the "12-month suicidal thought" and around 1% of all respondents for the "12-month suicidal attempt"). Moreover and because of their profiles, those individuals are more likely to have had a 12-month suicidal thought and/or a 12-month suicidal attempt which would have resulted in an underestimation of the prevalence. To fill in these missing responses, values were imputed using the approach described below.

Two methods of imputation were used, a deterministic method and one based on a logistic regression model. As it was possible to derive directly the missing value based on other responses for some respondents, a deterministic imputation method was first used. This was the case for all missing values for the 12-month suicidal attempt and for about one fourth of the missing values for the 12--month suicidal thought. For the remaining missing values of the 12--month suicidal thought, a logistic regression imputation method was used. The method consisted in fitting a logistic regression model between the variable to impute (the 12--month suicidal thought) and correlated characteristics using respondents without missing values who were similar to those to impute. Using the fitted model, a probability of response (yes or no) was calculated for each respondent who needed imputation; a response was then imputed based on that probability.

Estimation

The principle behind estimation in a probability sample such as Cycle 1.2 is that each person in the sample "represents", besides himself or herself, several other persons not in the sample. For example, in a simple random 2% sample of the population, each person in the sample represents 50 persons in the population. In the terminology used here, it can be said that each person has a weight of 50. The weighting phase is a step that calculates, for each person, his or her associated sampling weight. This weight appears on the microdata file, and must be used to derive meaningful estimates from the survey. For example, if the number of individuals who had a major depressive episode is to be estimated, it is done by selecting the records referring to those individuals in the sample having that characteristic and summing the weights entered on those records. In order for estimates produced from survey data to be representative of the covered population and not just the sample itself, a user must incorporate the survey weights into their calculations. A survey weight is given to each person included in the final sample, that is, the sample of persons having answered the survey. This weight corresponds to the number of persons represented by the respondent for the entire population.


List of adjustments in the weighting (refer to User Guide for more information):

0 Initial weight
1 Sample increase or decrease
2 Stabilization
3 Removal of out-of-scope units
4 Household non-response
5 Creation of person level weight
6 Person non-response
7 Post-stratification

In order to determine the quality of the estimate and to calculate the CV, the standard deviation must be calculated. Confidence intervals also require the standard deviation of the estimate.

The CCHS uses a multi-stage survey design, which means that there is no simple formula that can be used to calculate variance estimates. Therefore, an approximative method was needed. The bootstrap method is used because the sample design information needs to be taken into account when calculating variance estimates. The bootstrap method does this, and with the use of the Bootvar program, remains a method that is fairly easy for users to use.

The bootstrap re-sampling method used in the CCHS involves the selection of simple random samples known as replicates, and the calculation of the variation in the estimates from replicate to replicate. In each stratum, a simple random sample of (n-1) of the n clusters is selected with replacement to form a replicate. Note that since the selection is with replacement, a cluster may be chosen more than once. In each replicate, the survey weight for each record in the (n-1) selected clusters is recalculated. These weights are then post-stratified according to demographic information in the same way as the sampling design weights in order to obtain the final bootstrap weights.

The entire process (selecting simple random samples, recalculating and post-stratifying weights for each stratum) is repeated B times, where B is large. The CCHS typically uses B=500, to produce 500 bootstrap weights. To obtain the bootstrap variance estimator, the point estimate for each of the B samples must be calculated. The standard deviation of these estimates is the bootstrap variance estimator. Statistics Canada has developed a program that can perform all of these calculations for the user: the Bootvar program.

Quality evaluation

Survey design has a profound effect on the objectives of the survey which are listed under "Survey Description". To meet these objectives, a Steering Committee and an Advisory Board comprised of authorities from the provincial and territorial Ministries of Health, the Canadian Institute for Health Information and Health Canada determined the concepts and focus. An expert group was convened to advise on the measures to obtain the results envisioned by the Steering Committee and Advisory Board, and to recommend proven collection vehicles and indices. The resulting data is recognized as valid measures of contemporary concepts.

The frame chosen to provide the sample, the Labour Force Survey, has been combined with sampling design methodologies which have been tested, used repeatedly, and have been proven to produce accurate results. The large sample in each province helps ensure accurate and meaningful results.

Actions have been taken to reduce non-sampling errors to a minimum.

Further information about these measures and the quality of the data may be found in the CCHS public-use microdata documentation, available from the Health Statistics Division.

High response rates are essential for quality data. To reduce the number of non-response cases, the interviewers are all extensively trained by Statistics Canada, provided with detailed Interviewer Manuals, and are under the direction of interviewer supervisors. A maximum recommended assignment size by interviewer was calculated based on test results. Interviewers make every effort to contact respondents.

Refusals were followed up by senior interviewers, project supervisors or by other interviewers to encourage respondents to participate in the survey. In addition, to maximize the response rate, non-response cases were also followed up in subsequent collection periods.

The questionnaires were developed in consultation with Statistics Canada's Questionnaire Design Resource Centre and were reviewed and tested in the field in pre-tests and focus groups in both official languages. Questionnaires were also translated into 2 languages other than French and English to help ensure that language was not a barrier to obtaining quality data. Trained interviewers were available to interview respondents in Punjabi and Chinese.

Data was collected using the Computer Assisted Interview (CAI) system that has built-in edits and skip patterns which were extensively tested. The resulting data have been determined to provide accurate measures of the concepts being surveyed.

Coding of responses uses internationally recognized classifications such as the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM IV).

Medications were coded to the 2002 Anatomical Therapeutic Classification (ATC).

A Data Sharing Agreement provides the provinces and Health Canada with approved files for data analysis, which affords them the opportunity to help verify the efficacy of the collection vehicles and resulting data.

A user guide that includes Data Dictionaries is provided to all Public Use Microdata File users to help explain validity and reliability concepts, variance, and to aid in the analysis of the data. To account for survey design effects, standard errors and coefficients of variation may be estimated with the bootstrap technique.

Disclosure control

Statistics Canada is prohibited by law from releasing any data that would divulge information obtained under the Statistics Act that relates to any identifiable person, business or organization without the prior knowledge or the consent in writing of that person, business or organization. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Public Use Microdata Files (PUMFs) are produced in addition to the Master files. The PUMFs differ in a number of important aspects from the survey "master" files held by Statistics Canada. These differences are the result of actions taken to protect the anonymity of individual survey respondents. First, only cross-sectional data are available on such files, because longitudinal information can lead to the identification of respondents. Also, some sensible variables are regrouped, capped or completely deleted from the files. Users requiring access to information excluded from the microdata files may purchase custom tabulations, or access the master files through the Research Data Centres program or the Remote Access program. Outputs are vetted for confidentiality before being given to users.

Before releasing and/or publishing any estimate from these files, users should first determine the number of sampled respondents who contribute to the calculation of the estimate. If this number is less than 30, the weighted estimate should not be released regardless of the value of the coefficient of variation for this estimate. For weighted estimates based on sample sizes of 30 or more, users should determine the coefficient of variation of the rounded estimate and follow the guidelines below.

Estimates in the main body of a statistical table are rounded to the nearest hundred units using the normal rounding technique. If the first or only digit dropped is zero to four, the last digit retained is not changed. If the first or only digit dropped is five to nine, the last digit retained is raised by one. Marginal sub-totals and totals in statistical tables are derived from their corresponding unrounded components and then are rounded themselves to the nearest 100 units using normal rounding methods. Averages, proportions, rates and percentages are computed from unrounded components (for example, numerators and/or denominators) and then are rounded themselves to one decimal using normal rounding. In normal rounding to a single digit, if the final or only digit dropped is zero to four, the last digit retained is not changed. If the first or only digit dropped is five to nine, the last digit retained is increased by one. Sums and differences of aggregates (or ratios) are derived from their corresponding unrounded components and then are rounded themselves to the nearest 100 units (or the nearest one decimal) using normal rounding. Under no circumstances are unrounded estimates, published or otherwise, released. Unrounded estimates imply greater precision than actually exists.

Revisions and seasonal adjustment

This methodology does not apply to this survey.

Data accuracy

Most editing for the CCHS is conducted at the time of the interview by the Computer Assisted Interview (CAI) application. Some types of inconsistent or unusual reporting were edited after data collection at Head Office. Inconsistencies were usually corrected by setting answers to questions to 'not stated'.

Most of the reported information for Cycle 1.2 on mental health was left as is. Because of the potential sensitivity of the content, it was felt inappropriate to question the respondents on inconsistent answers during the interview and the information was left as collected on the file to allow researchers to deal with it as they see fit given that the answers could be subject to different interpretation.

A detailed Microdata User Guide was developed to provide all the relevant background information on the survey (background, methodology, data quality, data dictionary, derived variables specifications, etc).

Special studies were conducted on the survey data. These include a validation of CCHS results in relation to various other surveys.

For information on Response rates and Other accuracy issues, refer to this document.

• Additional documentation

Documentation

• Master File
• Public Use Microdata File