Canadian Internet Use Survey (CIUS)

Detailed information for 2018

The main objective of the 2018 Canadian Internet Use Survey (CIUS) is to gather information to better understand the adoption and use of digital technologies and the online behaviors of Canadians.

Data release - October 29, 2019

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

The 2018 Canadian Internet Use Survey (CIUS) measures access to the Internet and the online behaviours of individual residents of Canada 15 years of age and over, living in the provinces.

The survey is built off the previous iteration of the CIUS, last conducted in 2012. The 2018 iteration has been redesigned and modernized to increase international comparability, answer government policy-relevant questions, and measure a wider range of online activities, given the rapid pace at which the Internet has evolved.

The 2018 CIUS aims to measure the impact of digital technologies on the lives of Canadians. Information gathered will help to better understand how individuals use the Internet, including intensity of use, demand for online activities and online interactions. The CIUS examines, use of online government services, use of social networking websites or apps, smartphone use, digital skills, e-commerce, online work, and security, privacy and trust as it relates to the Internet.

Collected data is used to inform evidence-based policymaking, research and program development, and provide internationally comparable statistics on the use of digital technologies.

The 2018 iteration of the CIUS is sponsored by Innovation, Science and Economic Development Canada (ISED). Numerous other government departments also provided input during the questionnaire content development phase.

Reference period: Reference periods used are: "current or regular use", "past 3 months" or "past 12 months" preceding the interview date.

Subjects

• Individual and household internet use
• Information and communications technology

Data sources and methodology

Target population

The target population is all persons 15 years of age and older living in the ten provinces of Canada. It excludes full-time (residing for more than six months) residents of institutions.

People living on reserves are not excluded from the survey population, but they are not targeted in the sample. As the CIUS does not collect this characteristic, there is no way to identify whether the respondents, who are selected from a telephone-based frame, reside on a reserve.

Instrument design

The predecessor to the CIUS, the Household Internet Use Survey (HIUS), was first conducted in 1997, and ran annually until 2003. The HIUS focused on household Internet penetration. In 2005, the CIUS replaced the HIUS. The redesign in 2005 focused more on Internet use by individuals, while conforming to international standards regarding statistical indicators for Internet access and use. From 2005 to 2009, the CIUS was conducted biennially.

In 2010, the CIUS was redesigned to meet the measurement needs of Broadband Canada: Connecting Rural Canadians Program sponsored by Industry Canada (now ISED). As a hybrid survey on access and use, the CIUS measured the type, speed and cost of household Internet access and the individual online behaviours of a selected household member.

In the 2018 cycle of CIUS, the questionnaire content was revised through consultations with the clients at ISED, subject matter experts, other federal departments, and external stakeholders. While there is some comparability with the 2012 CIUS, the 2018 survey was redesigned to reflect the rapid pace at which Internet technology has evolved since the previous survey iteration. These indicators measure a range of online behaviours, including those related to social media, e-commerce, e-government, peer-to-peer services and content streaming.

Testing of the questionnaire:
Cognitive testing of the questionnaire content was carried out in two phases in conjunction with the Questionnaire Design Resource Center based at Statistics Canada in both official languages. The first round of testing concentrated on validating respondents' understanding of concepts, questions, terminology, the appropriateness of response categories and the availability of requested information. The content was tested through one-on-one interviews with 24 potential respondents that took place in Ottawa, Arnprior and Halifax. The resulting comments and analysis of these interviews led to further revision of the questionnaire to make the questions more relevant to respondents and easier to answer.

For the second phase, a total of 23 one-on-one cognitive interviews took place in Montreal and Edmonton to assess the updated questionnaire content (based on the results of the previous round of testing) using mock-up screen shots in an updatable PDF form to simulate an electronic questionnaire (EQ). This final round of testing confirmed that respondents could navigate through the EQ application with ease, while providing the requested information.

The primary challenge in developing the questionnaire was to balance the data needs of the users with the ability and willingness of potential respondents to provide the information.

Questionnaire testing revealed that certain technology-related terms were more commonly understood in English than in French among the francophone respondents consulted. As a result, certain English terms were added in parentheses following the use of the French term in the French questionnaire to ensure that the concepts were understood by the largest possible number of respondents.

Sampling

This is a sample survey with a cross-sectional design.

The sample is no longer taken from the Labour Force Survey (LFS) sample.

This survey uses a frame that combines landline and cellular telephone numbers from the Census and various administrative sources with Statistics Canada's dwelling frame. Records on the frame are groups of one or several telephone numbers associated with the same address (or single telephone number in the case a link between a telephone number and an address could not be established). This sampling frame is used to obtain a better coverage of households with a telephone number.

The sample is based on a stratified design employing probability sampling. The stratification is done at the province level. Information is collected from one randomly selected household member aged 15 or older, and proxy responses are not permitted.

Sampling unit:
The CIUS uses a two-stage sampling design. The sampling units are the groups of telephone numbers. The final stage units are individuals within the identified households. Note that the CIUS only selects one eligible person per household to be interviewed.

For the survey, a single eligible member of each sampled household is randomly selected.

A field sample of approximatively 33,000 units was used. Among them, about 26,000 had addresses that were attached to the telephone number in the frame. Invitation letters to complete the electronic questionnaire were sent to these selected households across Canada. The remaining portion of the sample that did not receive an invitation letter because they did not have an address attached to their telephone number, were contacted by telephone to complete the questionnaire with an interviewer. A completion of 15,000 questionnaires was expected.

Data sources

Data collection for this reference period: 2018-11-15 to 2019-03-21

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

Data are collected through an electronic questionnaire or computer assisted telephone interviewing (CATI). No proxy reporting is allowed. The respondent has the choice to respond in French or in English. The average time to complete the survey is estimated at 30-60 minutes.

Invitation letters and reminders, containing secure access code, to complete electronic questionnaires were sent to respondents by mail. Intensive non-response follow-up was also conducted by CATI.

The collection method has changed from telephone, solely, in 2012, to electronic questionnaire for 2018, with CATI follow-up.

View the Questionnaire(s) and reporting guide(s) .

Error detection

All responses to the 2018 CIUS questions were captured directly in the EQ application, both for the interviewer-led (iEQ) component and the respondent self-reporting (rEQ) component. The EQ application, just like any other computerized questionnaire, reduces processing time and costs associated with data entry, transcription errors and data transmission.

For some CIUS questions, data underwent a preliminary verification process when respondents were completing the survey. This was accomplished by means of a series of edits programmed into the EQ. That is, where a particular response appeared to be inconsistent with previous answers, the interviewer or the self-reporting respondent was notified with an on-screen warning message, providing them with an opportunity to modify the response provided.

Once the data are collected, an extensive series of processing steps are undertaken including the editing and imputation process to identify inconsistent or missing data, and to correct errors. For example, these steps consist of a top-down flow edit to correct questionnaire paths mistakenly followed.
Abnormally large reported values for household income and for the value of the amount spent on physical goods and on digital goods or services were identified as "outliers" and treated by replacing the suspicious values by ones from respondents with similar characteristics (see Imputation).

Imputation

Imputation is the process that supplies valid values for those responses that have been identified as either invalid or missing on the data file. The new values are supplied in such a way as to preserve the underlying structure of the data and to ensure that the resulting records will pass all required edits. Imputation was done using a nearest-neighbor method which searches for "donor" records from individuals with complete and consistent values. The recipient records are imputed by a donor chosen from a group of records with similar demographic characteristics.

CIUS imputation was limited to item non-response for the sex of the respondent, the household income and for the value of the amount spent on physical goods over the Internet and on digital services over the Internet. Imputation was used for the rare cases where the information was missing regarding the sex of a respondent. The data was examined in details and useful information such as the age, the sex of the other members of the household, the first name and the interviewer's comments were taken into account for imputing this variable.

In the case of income, in total, about 10,000 respondents (71%) reported their household income. Respondents who did not provide a dollar estimate of their income were asked for an income range. About 2,000 respondents (14%) did not provide any information on their income. The reported income ranges and the missing income information were imputed by donor values in a series of steps, depending on the available information.

For the value of the amount spent on physical goods and on digital goods and services over the Internet, donor imputation was also used. Again, the donor records were chosen from a group of records with similar demographic characteristics, as well as similar Internet shopping behavior (e.g. types of goods and services). The relative imputation rate serves as a data quality indicator. The rates for 2018, based on value-weighted estimates are the following: for every $100 spent in online orders for physical goods estimated from the survey, about $7.14 orders were imputed. Similarly with the value of orders for digital goods and services, for every $100 spent in online orders for digital services estimated from the survey, $2.98 was imputed.

Estimation

When a probability sample is used, as it was the case for this survey, the principle behind estimation is that each person selected in the sample represents (in addition to himself/herself) several other persons not in the sample. For example, in a simple random sample of 2% of the population, each person in the sample represents 50 persons in the population (himself/herself and 49 others). The number of persons represented by a given respondent is usually known as the weight or weighting factor.

The 2018 CIUS is a survey of individuals and the analytic files contain questionnaire responses and associated information from the respondents.

A weighting factor is available on the microdata file:
WTPM: This is the basic weighting factor for analysis at the person level, i.e. to calculate estimates of the number of persons (non-institutionalized and aged 15 or over) having one or several given characteristics.
In addition to the estimation weights, bootstrap weights have been created for the purpose of design-based variance estimation.

Estimates based on the survey data are also adjusted (by weighting) so that they are representative of the target population with regard to certain characteristics (each month we have independent estimates for various age-sex groups by province). To the extent that the characteristics are correlated with those independent estimates, this adjustment can improve the precision of estimates.

Quality evaluation

Rigorous quality assurance mechanisms are applied at all stages of the statistical process, including questionnaire design, data collection and data processing. Measures included testing of the questionnaire with potential respondents, training provided to interviewers for specific survey concepts and procedures and observations of interviews. Finally, a validation of the results was implemented to ensure that the data is consistent with other previously published data. To do this, estimates were produced in the form of cross-tabulations and then compared with other similar data sources such as the 2012 CIUS cycle, the Digital Economy Survey, (record number 5265), regulatory agencies (e.g. the Canadian Radio-television and Telecommunications Commission), and international organizations (e.g. the OECD).

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Revisions and seasonal adjustment

This methodology does not apply to this survey program.

Data accuracy

As the data are based on a sample of persons, they are subject to sampling error. That is, estimates based on a sample will vary from sample to sample, and typically they will be different from the results that would have been obtained from a complete census. More precise estimates of the sampling variability of estimates can be produced with the bootstrap method using bootstrap weights that have been created for this survey. The bootstrap method was used to estimate the sampling variability for all of the estimates produced based on the data from 2018 CIUS.

The sample error is quantified by the coefficient of variation (CV) with the following guidelines:
- 16.5% and below: Acceptable estimate;
- 16.6% to 33.3%: Marginal estimate, with cautionary note;
- Above 33.3%: Unacceptable estimate.

Estimates that do not meet an acceptable level of quality are either flagged for caution or suppressed.

Response rate:
The overall response rate is 43.7%.

Non-sampling error:
Common sources of these errors are imperfect coverage and non-response. Coverage errors (or imperfect coverage) arise when there are differences between the target population and the surveyed population. Households without telephones, as well as households with telephone services not covered by the current frame, represent a part of the target population that was excluded from the surveyed population. To the extent that the excluded population differs from the rest of the target population, the results may be biased. In general, since these exclusions are small, one would expect the biases introduced to be small. Survey estimates will be adjusted (i.e. weighted) to account for non-response cases. Other types of non-sampling errors can include response errors and processing errors.

Non-response bias:
The main method used to reduce non-response bias involved a series of adjustments to the survey weights to account for non-response as much as possible.

Coverage error:
The survey frame for CIUS was created using several linked sources, such as the Census, administrative data and billing files. All respondents in the ten provinces were interviewed by telephone or self-completed an electronic questionnaire. Households without telephones were therefore excluded from the survey population. Survey estimates were adjusted (weighted) to represent all persons in the target population, including those not covered by the survey frame.

Other non-sampling errors:
Significant effort was made to minimize bias by using a well-tested questionnaire, a proven methodology, specialized interviewers and strict quality control.