Childhood National Immunization Coverage Survey (CNICS)

Detailed information for 2017

The purpose of the Childhood National Immunization Coverage Survey is to collect information on national immunization coverage for childhood vaccines.

Data release - March 26, 2019

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

The purpose of the Childhood National Immunization Coverage Survey (CNICS) is to collect information on national immunization coverage for childhood vaccines. The survey is intended to:

- Determine if children are immunized in accordance with recommended immunization schedules for publicly-funded vaccines
- Provide the World Health Organization and the Pan American Health Organization with estimates of national vaccine coverage for childhood vaccines such as measles, diphtheria, pertussis, tetanus and polio
- Provide information on parental knowledge, attitudes, and beliefs about vaccines.

Reference period: Varies

Collection period: From fall to spring

Subjects

• Children and youth
• Health

Data sources and methodology

Target population

The target population for this survey is children aged 2, 7, 14 and 17 years old, on March 1st, 2017, living in the 10 provinces and three territories, not residing on First Nations reserves and not institutionalized.

Instrument design

The survey content of the 2011 CNICS was developed in collaboration with the Public Health Agency (PHAC). The survey content was tested by the Questionnaire Design Research Centre (QDRC) of Statistics Canada in one-on-one interviews in participants' homes prior to collection. There were no significant content changes in 2013 and 2015. New survey content was developed in 2017 and tested by the QDRC in one-on-one interviews in testing facilities in Vancouver, Montreal and Ottawa.

Data collection for the CNICS is comprised of multiple components. The first component is a computer assisted telephone interview (CATI) during which the person most knowledgeable about the child's immunizations (PMK), provides information about vaccines the child has received and answers questions regarding their knowledge, attitudes and beliefs about immunization. The completion of the telephone interview triggers the second component, a mail out/mail back consent form, which is sent to respondents who have agreed to provide written consent to Statistics Canada to contact the child's health care provider(s) to obtain their child's vaccine history. The third component is the Immunization Record Request form, a mail out/mail back questionnaire sent to the child's health care provider(s) who fill(s) out and returns the form with the child's detailed immunization history. The data from the health care providers are used to enhance the PMK reported data.

The 2017 CNICS asked respondents for permission to access their child's vaccination record from regional, provincial and territorial immunization registries. Where available, the linked data will be used to evaluate the potential to integrate immunization registry data in future cycles of the survey. In order to access this information, Statistics Canada must enter into data sharing agreements and data acquisition agreements with each individual jurisdiction. Linkage will only be undertaken for consenting respondents.

Sampling

This is a sample survey with a cross-sectional design.

The main objective of the 2017 CNICS is to produce provincial and territorial estimates of immunization coverage for children age 2 years old and national estimates of immunization coverage for children and youth aged 7, 14 and 17 years old. The target population is stratified by age and by province and territory to ensure that the sample is representative while remaining efficient. It was determined that a sample of 14,960 units would yield the desired accuracy.

Data sources

Data collection for this reference period: 2017-11-20 to 2018-04-27

Responding to this survey is voluntary.

Data are collected directly from survey respondents, who are the parent or guardian of the selected child.

Data are collected using computer-assisted telephone interviewing (CATI).

With consent, data are also collected from the selected child's immunization provider (e.g. physician, nurse, or clinic). The data collected from the immunization provider is combined with the immunization data collected from the parent or guardian.

Immunization data is also obtained from the regional, provincial and territorial immunization registries when the CNICS respondent has given consent to link to and access their child's vaccination record. This step only happens for jurisdictions with whom Statistics Canada has concluded data sharing and data acquisition agreements.

View the Questionnaire(s) and reporting guide(s) .

Error detection

The purpose of processing survey data is to adapt the collected data into a form that is appropriate for analysis and tabulation.

For the CNICS, collection was performed using a Computer Assisted Telephone Interview (CATI), which allows for certain edits to be built into the application. For example, Validity Edits, which ensure that the response falls within the allowed range. It also ensured that only character values were entered into character fields or numeric values were entered into numeric fields.

After collection the raw data file was put through a series of standard processing steps designed to clean the data and help ensure its consistency thereby increasing its usefulness. The edits were done on the data both at the micro and macro level.

The flow edits replicated the flow patterns used in the application and set the non-applicable questions to a value of 'Valid Skip'. Non-responses were set to a value of 'Not Stated'. These are questions that were applicable to the respondent but were not answered. In a CATI application these value usually follow a response of 'Refusal' or 'Don't Know'.

In addition, various types of editing were done to detect missing or inconsistent information. For example, edits were performed to check the logical relationship between responses.

New variables were derived using collected variables. A derived variable may be based on one variable by re-grouping or collapsing the categories or based on several variables, by combining them together to define a new concept.

Imputation

Total household income was imputed when it was not provided by the respondent during the interview. Income imputation was done using the trend imputation method or, when this was not possible, the nearest-neighbour imputation method. The nearest-neighbour method finds and uses the income data from a donor respondent who shares similar characteristics to the recipient respondent.

In addition, some imputation for tetanus, diphtheria and pertussis immunization was done to account for the fact that these vaccines are only available in combination (Tdap or Tdap-IPV) in Canada.

Estimation

The Childhood National Immunization Coverage Survey (CNICS) is a probability survey. As is the case with any probability survey, the sample is selected so as to be able to produce estimates for a reference population. Therefore, each unit in the sample represents a certain number of units in the population.

Survey weights are calculated by taking the child's design weight and making adjustments for survey non-response and post-stratification to ensure that the final survey weights sum to known counts of children by age. The design weight is the inverse of the probability of selection, that is, the probability that a child in the population is selected into the CNICS sample.

First adjustment: Non-response adjustment
Weights were adjusted in order that the respondents also represent the non-respondents. This ensures population totals are not underestimated. The method of response homogeneous groups (RHGs) was applied to assign the weight of the non-respondents to the respondents. The RHG method involves grouping together individuals with the same likelihood of response and then computing an adjustment factor for each RHG.

Second adjustment: Post-stratification
The last adjustment ensures consistency between the estimates produced by CNICS and Statistics Canada's population estimates by age and province. This method is called post-stratification. The purpose of this adjustment is to ensure that the sum of the weights match known population totals.

Numbers used in the post-stratification refer to the population counts on March 1st, 2017, as estimated by Statistics Canada.

Two sets of weights were calculated for CNICS. The first set is used to estimate the coverage for all vaccines while the second must be used to estimate knowledge, attitudes and beliefs about vaccines for the population. Both sets were calculated using the aforementioned methodology.

To estimate variances directly, two sets of 1,000 bootstrap weights were also created and made available in separate files.

Quality evaluation

While rigorous quality assurance mechanisms are applied across all steps of the statistical process, validation and scrutiny of the data by statisticians are the ultimate quality checks prior to dissemination. Many validation measures were implemented. They included (among others):
a) Analysis of changes over time (where possible)
b) Verification of estimates through cross-tabulations
c) Consultation with stakeholders internal to Statistics Canada
d) Consultation with the Public Health Agency of Canada
e) Review of production processes
f) Coherence analysis based on quality indicators

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

In order to keep the data confidential, some modifications are performed on the collected data.

Revisions and seasonal adjustment

This methodology does not apply to this survey program.

Data accuracy

As the data are based on a sample of persons, they are subject to sampling error. That is, estimates based on a sample will vary from sample to sample, and typically they will be different from the results that would have been obtained from surveying the entire population.

The quality of estimates produced with CNICS data is measured with the confidence intervals (CI), produced using bootstrap weights. The CI range will depend on the domain of interest and the prevalence of the characteristic.

The response rates for the CNICS are shown in the table below - see additional documentation. The response rates reported here are the number of children whose parents responded to the questions in the immunization section or the questions in the knowledge, attitudes and beliefs section of the CATI survey as a percentage of the number of cases sampled.

The overall unweighted response rate was 60.6% in 2017. The response rates according to the age group ranged between 48.0% and 67.4%.

Much time and effort was devoted to reducing non-sampling errors in the survey by using a well-tested questionnaire, a proven survey methodology, specialized interviewers and quality control measures.

• Additional documentation