Childhood National Immunization Coverage Survey (CNICS)

Detailed information for 2011

The purpose of the Childhood National Immunization Coverage Survey is to collect information on national immunization coverage for childhood vaccines.

Data release - December 3, 2012

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

The purpose of the Childhood National Immunization Coverage Survey (CNICS) is to collect information on national immunization coverage for childhood vaccines. The survey is intended to:

- Determine if children are immunized in accordance with recommended immunization schedules for publicly-funded vaccines
- Provide the World Health Organization and the Pan American Health Organization with estimates of national immunization coverage for diphtheria-pertussis-tetanus (DPT) and for measles-mumps-rubella (MMR) and other vaccines for children
- Assess knowledge, attitudes, and awareness of vaccines.

Reference period: Varies according to the question

Collection period: September to December

Subjects

• Children and youth
• Health
• Prevention and detection of disease

Data sources and methodology

Target population

The target population for this survey is children aged 2, 7, 10 to 14 (girls) and 17 years old, living in the 10 provinces and three territories, not residing in institutions or reserves.

Instrument design

The content was developed in coordination with the Public Health Agency (PHAC). The survey content was tested by the Questionnaire Design Research Centre of Statistics Canada in one-on-one interviews in participants' homes prior to collection.

Data collection for the CNICS is comprised of three components. The first component is a computer assisted telephone interview (CATI) during which the parent answers survey questions about their child's immunizations. The completion of the telephone interview triggers the second component, a mail out/mail back Parent Consent form, which is sent to respondents who agreed at the time of the interview to give written consent to Statistics Canada to contact their immunization provider(s). The third component is the Immunization Record Request form, a mail out/mail back questionnaire sent to the child's immunization provider(s) who fills it out with the child's immunization history (name of vaccination and date given). The data provided by the immunization providers are used to enhance the parent reported data.

Sampling

This is a sample survey with a cross-sectional design.

The main objective of this survey is to produce estimates of coverage for childhood vaccines for children aged 2, 7, 10-14 (girls) and 17 years old at the national level. The target population is stratified by age and by province and territory to ensure that the sample is representative while remaining efficient. It was determined that a sample of approximately 2,800 units would yield the desired accuracy.

Respondents were randomly selected from an administrative file available at Statistics Canada. The sampling unit is the child, but the respondent is their parent or guardian; as such, to reduce response burden, the design prohibits two children from the same household from being selected for the sample.

Data sources

Data collection for this reference period: 2011-09-26 to 2011-12-24

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

Parents/guardians serve as proxy respondents for children of all age groups.

The questionnaire is administered by an interviewer using computer-assisted interviewing (CAI). All interviews are conducted by telephone (CATI). At the beginning of the interview the Person Most Knowledgeable (PMK) about the child's immunizations is identified.

A consent form is sent to respondents who have given permission to Statistics Canada to contact their child's immunization providers.

An immunization record request form is used to collect data from immunization providers.

A tracking system is used to identify respondents who consent to the immunization provider contact and consent forms are immediately printed and sent to the respondents for their signature. The tracking system records when the signed forms are returned, which then initializes the mail-out to the immunization providers. The tracking system is also used for follow-up.

View the Questionnaire(s) and reporting guide(s) .

Error detection

Edits were developed as part of the data capture application. These edits are applied at the time of collection to ensure data quality. Anomalies in the information reported are confirmed with the respondent right away and corrected if necessary.

For the computer-assisted telephone interviewing (CATI) application, it is not possible for interviewers to enter out-of-range values, and flow errors are controlled through programmed skip patterns. For example, CATI ensures that questions that do not apply to the respondent are not asked. In response to certain types of inconsistent or unusual reporting, warning messages are invoked. In some instances, no corrective action is taken at the time of the interview and edits are instead performed at Head Office after data collection.

Imputation

The only variable that was imputed is the total household income. An imputation flag has been included on the CNICS file so that users will have information on the extent of imputation.

Income imputation was done using the nearest-neighbour imputation method. The first step of this method is to find a respondent (the donor) who answered the income section and has characteristics similar to those of the respondent who did not provide complete income information (the recipient). The donor's record is then used to calculate the values to be imputed to the recipient's record.

Estimation

The Childhood National Immunization Coverage Survey (CNICS) is a probability survey. As is the case with any probability survey, the sample is selected so as to be able to produce estimates for a reference population. Therefore, each unit in the sample represents a number of units in the population.

Survey weights are calculated by taking the child's design weight and making adjustments for survey non-response and post-stratification to ensure that the final survey weights sum to known counts of children by age. The design weight is the inverse of the probability of selection, that is, the probability that a child in the population is selected into the CNICS sample.

First adjustment: Non-response adjustment
We needed to adjust the weights so that the respondents represent the non-respondents. Otherwise, we would, for example, underestimate totals. To decide how to assign the weight of the non-respondents to the respondents, we apply the method of response homogeneous groups (RHGs). The RHG method involves grouping individuals with the same likelihood of response. Then an adjustment factor is computed for each RHG. In the first step, the weights of the cases that we were not able to contact were given to cases for which a contact was made (respondents and other non-respondents) while with the second step, the weights of the other non-respondents were given to the respondents.

Second adjustment: Post-stratification
The last adjustment factor ensures consistency between the estimates produced by CNICS and Statistics Canada's population estimates by age and province. This method is called post-stratification. The purpose of this adjustment is to ensure that the sum of the weights match known population totals

Numbers used in the post-stratification refer to the population counts on December 31, 2011, as estimated by Statistics Canada.

Because of a high percentage of partial non-response for a component of the survey, two weights were calculated for CNICS. The first weight could be used to estimate the coverage for all vaccines while the other weight could be used to assess knowledge, attitudes and awareness of vaccines. Both weights were calculated using the aforementioned methodology.

Sampling variance calculation

It would be difficult (not to say impossible) to derive an exact formula to calculate the sampling variance for CNICS because of the sample design, non-response adjustments, treatment of out-of-scope units and post-stratification. Actually, such a task could only be undertaken under such strong assumptions as to yield a framework too simplistic to be of any use in practice. One way to approximate the sampling variance is to use the bootstrap method. With that method, we generate a set of 1,000 weights, known as bootstrap weights, which are derived from the survey weights and used to estimate the variance of the estimates. Two sets of 1,000 bootstrap weights are available for CNICS on a separate data file.

Quality evaluation

Training
The CNICS is conducted by Statistics Canada interviewers. Project managers and senior interviewers are responsible for ensuring that the CNICS interviewers are familiar with the survey's concepts and procedures.

Refusals
In the event a respondent refused to be interviewed, Statistics Canada interviewers are trained basic refusal conversion techniques. If a respondent is adamant, the interviewer is instructed to obtain as much information as possible about the respondent (such as why they are refusing to participate) and refer the case to the senior interviewer. The senior interviewer then attempts to contact the respondent and convert the case. If the senior interviewer is unable to do so, a letter is sent to the respondent as a final effort to convert the case.

Tracing
In the event a household could not be reached due to incorrect/outdated contact information from the sample file, the case was transferred to tracing. Attempts were then made to find the correct contact information using various means.

The survey applications
The use of CAI allows for complex flows and edits to be built into the questionnaire, helping with data quality and ensuring that respondents answer only the questions appropriate to their situations. The survey application underwent testing at Statistics Canada to ensure that it functioned properly. During collection, review screens, range edits, flow pattern edits and general consistency edits, were used for quality control.

Processing
Once the data were collected, they underwent processing steps to check the quality of the data. The steps were pre-edit, flow edit and consistency edit. After the consistency edit step, the derived variables were checked to ensure that they were programmed correctly and that all records were assigned their correct values.

The coverage rates of the vaccines included on the CNICS were compared to a past survey that had similar content to ensure that the coverage rates were consistent with previous findings. The data provided by the health care providers were used to assess the accuracy of the parent reported data.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects that could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

In order to prevent any data disclosure, confidentiality analysis is done using the Statistics Canada Generalized Disclosure Control System (G-Confid). G-Confid is used for primary suppression (direct disclosure) as well as for secondary suppression (residual disclosure). Direct disclosure occurs when the value in a tabulation cell is composed of or dominated by few enterprises while residual disclosure occurs when confidential information can be derived indirectly by piecing together information from different sources or data series.

Revisions and seasonal adjustment

This methodology does not apply to this survey.

Data accuracy

The survey population is built using the list of applicants to the Canadian Child Tax Benefit (CCTB), which is a monthly file provided by the Canada Revenue Agency (CRA).The difference between the target and survey populations consists of the children for whom no parent or guardian applied for CCTB benefits. This may include families who are not aware of the benefit, or who chose not to request it. It may also include children who lived with a foster family for the entire year, as their caregivers would have been subsidized by the provincial government. The coverage of the CCTB was estimated to be around 96% in 2009, by comparing with demographic projections that were based on the 2006 Canadian Census of Population. Analyses revealed no important undercoverage bias.

The overall response rate is 64.9%. The response rates by age group range between 63% and 67.6%.