Canadian Armed Forces Health Survey (CAFHS)

Detailed information for 2019

The objective of this survey is to provide insight into the current health status of Canadian Armed Forces members.

Data release - December 4, 2019

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

Results from this survey will be used to:
- guide the development and evaluation of policies and programs aimed at sustaining a healthy and deployable force;
- provide a snapshot of the health status of DND-CAF members;
- identify where interventions are most needed to sustain a healthy and readily deployable force.

Reference period: Most questions refer to the previous 12-month period

Collection period: January to June 2019

Subjects

• Diseases and health conditions
• Health
• Health care services
• Lifestyle and social conditions

Data sources and methodology

Target population

The target population for this survey includes actively serving Department of National Defence Canadian Armed Forces (DND-CAF) members from both the Regular Force and Class A, B and C Reserve Force. Canadian Rangers, Supplementary Reservists and Cadets are excluded.

Instrument design

Representatives from the Directorate Force Health Protection (DFHP) within the Department of National Defence (DND) worked with Statistics Canada to develop the survey content. The majority of the content was based on content from the Health and Lifestyle Information Survey (HLIS), which was previously administered by DND. Content from the Canadian Community Health Survey (CCHS) was also incorporated.

The questionnaire went through qualitative testing to ensure that survey respondents understood the questions and to identify areas of improvement. Cognitive interviews were administered to recruited test participants in Kingston, Victoria and Montreal from the target population for the survey. This process was led by the Questionnaire Design Resource Centre of Statistics Canada.

After testing, the survey content was finalized in collaboration with the survey sponsors. An electronic questionnaire application was then developed, where the content was programmed into an integrated electronic questionnaire application.

Sampling

This is a sample survey with a cross-sectional design.

A stratified random sample is selected of actively serving Department of National Defence Canadian Armed Forces (DND-CAF) members from both the Regular Force and Class A, B and C Reserve Force.

Sampling unit:
The sampling units consist of Regular Force and Reserve Force members.

Stratification method:
For Regular Force, the population strata are base, age group and sex. A base must meet the minimum threshold of consisting of 400 members in order to be a stratum. For Reserve Force, the population strata are provincial region, age group and sex.

Sampling and sub-sampling:
The number of respondents to be sampled is 33,000 actively serving Regular Force members and 4,000 Class A, B and C Reserve Force members. Half of the respondents will be assigned to wave 1 and half will be assigned to wave 2. Canadian Rangers, Supplementary Reservists and Cadets are excluded from the population.

Data sources

Data collection for this reference period: 2019-01-21 to 2019-06-30

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

Data were collected using an electronic questionnaire (EQ). The EQ application included a standard set of response codes to identify all possible outcomes and include soft edits to check the consistency of responses for some questions. The EQ application was tested prior to use to ensure that non-valid question responses were flagged and that all question flows produced the correct result. These measures helped to limit the amount of response data that had to be "cleaned" at the end of the collection process.

In order to introduce the agency, the name and purpose of the survey, the collaboration with the Department of National Defence Canadian Armed Forces, and how the survey results would be used, the electronic application began with an introduction page. Persons were told that their participation in the survey is voluntary, and that their information would remain strictly confidential. Respondents were also informed that to enhance the data from this survey and to reduce the reporting burden, Statistics Canada may combine the information with other survey or administrative data sources.

Prior to collection, Statistics Canada sent an introductory letter to each person with a valid mailing address via Canada Post and an email invitation to each person who had an available work email address. Mail and email reminders were sent multiple times during collection.

Responses to survey questions were captured directly by the secure network when respondents submitted questionnaires. Electronic questionnaires were considered more appropriate for sensitive content, and could also reduce transcription errors and data transmission. The response data were encrypted to ensure confidentiality and transferred over a secure network for further processing.

View the Questionnaire(s) and reporting guide(s) .

Error detection

Some editing was done directly at the time of collection. Where the information entered is out of range (too large or small) of expected values, or inconsistent with the previous entries, the respondent is prompted, through messages on the electronic questionnaire, to modify the information. However, for some questions respondents have the option of bypassing the edits, and of skipping questions if the respondent does not know the answer or refuses to answer. Therefore, the collected data are subjected to further edit processes once they arrive in head office.

The editing during processing consist, first of all, of identifying and eliminating potential duplicate records and identifying non-response and out-of-scope records. Second, consistency is ensured by modifying the data at the individual variable level. Third, the items from the survey output which need to be kept on the survey master file were determined. Fourth, invalid characters were deleted and the data items were formatted appropriately. Fifth, text fields were stripped off the main files and written to a separate file for coding.

Imputation

This methodology type does not apply to this statistical program.

Estimation

When a probability sample is used, as it was in the case of this survey, the principal behind estimation is that each person selected in the sample also represents other people not selected in the sample. For example, in a simple random sample of 5% of the population, each person in the sample represents 20 people in the population. The number of people represented by a given respondent is known as the survey weight.

The analytic files associated with this survey contain questionnaire responses and survey weights (i.e. the variable WTPM). In addition to the estimation weights, bootstrap weights have been created for the purpose of design-based variation estimation.

Quality evaluation

While rigorous quality assurance mechanisms are applied across all steps of the statistical process, validation and detailed review of the data are the ultimate quality checks prior to dissemination. Many validation measures were implemented.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects that could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Revisions and seasonal adjustment

This methodology type does not apply to this statistical program.

Data accuracy

As a health survey, we collected information on many facets of health. For example, we asked questions about chronic conditions such as asthma and high blood pressure. Overall, the survey estimates that 3.9% of CAF members had asthma and 8.7% had high blood pressure. The associated Coefficient of Variation (CV) for these estimates was 5.8% and 3.7%, respectively.

One of the objectives of the survey was to get estimates for smaller domains such as estimates by age group and sex. For the Regular Force, that meant obtaining estimates at the Canadian Forces base level. For Reservists, it meant estimates at the regional level.

Software used
EQGS4.1 portal for collection. SAS for processing.

Response rates
The overall response rate was 38.4%. The response rate was 39.9% for the Regular Force and 26.4% for the Reserve Force.

Non-sampling error
Common sources of these errors are imperfect coverage and non-response. Coverage errors arise when there are differences between the target population and the surveyed population. Canadian Armed Force members who were part of the Forces but not on the file represent a part of the target population that was excluded from the surveyed population. To the extent that the excluded population differs from the rest of the target population, the results may be biased. Survey estimates were adjusted (i.e. weighted) to account for non-response. Other types of non-sampling errors include response errors and processing errors.

Non-response bias
We made adjustments to the survey weights to reduce potential non-response bias. The adjustments were made by sex and age groupings within the Regular and Reserve Forces.

Coverage error
Coverage errors consist of omissions, erroneous inclusions, duplications and misclassifications of units in the survey frame. Coverage errors may cause a bias in the estimates and the effect can vary for different sub-groups of the population. This is a very difficult error to measure or quantify accurately.

The survey frame was created using files extracted from administrative files coming from the Canadian Armed Forces. Unfortunately, there were no other sources which would allow us to assess the coverage of our population of interest. However, in the interest of reducing potential omissions or erroneous inclusions, the CAFHS sample frame was drawn as close to the start of collection as possible; one month before collection.

Other non-sampling errors
When talking about other non-sampling error we are referring to two sources of error: response and processing.
A response error occurs when the response provided differs from the real value. This could occur if the respondent doesn't understand the question, doesn't remember the actual value, accidentally types in the incorrect information or the information is not properly captured. In order to reduce the level or response errors the CAFHS underwent a questionnaire review and testing using cognitive methods. Whenever possible content that had been already tested and used in other Statistics Canada surveys was incorporated into the CAFHS questionnaire. The electronic questionnaire was also tested to ensure the questions appeared in the correct order and the data entered was captured properly.

Processing error is the error associated with activities conducted once survey responses have been received. It includes all data handling activities after collection and prior to estimation. Like all other errors, they can be random in nature, and inflate the variance of the survey's estimates, or systematic, and introduce bias. It is difficult to obtain direct measures of processing errors and their impact on data quality especially since they are mixed in with other types of errors (non-response, measurement and coverage).

Data processing of the CAFHS was done in a number of steps including data capture, verification, coding and editing. At each step of data processing, the data before and after the process is compared. Any errors detected are reported back to the processing team for correction.