Canadian Survey of Cyber Security and Cybercrime (CSCSC)

Detailed information for 2022

The purpose of the Canadian Survey of Cyber Security and Cybercrime is to measure the impact of cybercrime on Canadian businesses. For reference year 2022, the survey will be known as the Public Sector Survey on Cyber Security and Cybercrime (PSSCSC) with a focus on the Canadian public service and crown corporations.

The survey gathers information about:
- The measures organizations have implemented for cyber security, including employee training;
- The types of cyber security incidents that impact organizations; and
- The costs associated with preventing and recovering from cyber security incidents.

Data release - October 18, 2023

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

The 2022 Public Sector Survey on Cyber Security and Cybercrime is conducted on behalf of Public Safety Canada. This survey was launched because of the need to benchmark and monitor the rapidly evolving environment surrounding cyber security and cybercrime. The data collected serves the following broad objectives: to further understand the impact of cybercrime on the Canadian public service and crown corporations, including aspects such as investment in cyber security measures, cyber security training, the volume of cyber security incidents and the costs associated with responding to these incidents.

Reference period: The 12-month calendar year

Collection period: January through March

Subjects

• Business and government internet use
• Crime and justice
• Information and communications technology

Data sources and methodology

Target population

The target population was derived from Statistics Canada's Business Register (BR). The BR is an information database on the Canadian business population and serves as a frame for all Statistics Canada business surveys. It is a structured list of businesses engaged in the production of goods and services in Canada.

For the 2022 Public Sector Survey on Cyber Security and Cybercrime, the target population will consist of organizations from Schedule IV and V of the Financial Administration Act (FAA) and federal government units from the 2020 Public Sector Universe (PSU). The in-scope units from the PSU include federal government business enterprises but exclude the unit which represents the Government of Canada as a whole.

Instrument design

The survey data are collected using an electronic questionnaire.

In 2022, the questionnaire had minor revisions made in order to better meet the policy needs of the sponsoring partner Public Safety Canada, as well as to ensure the questionnaire content was relevant to the new target population. Subject matter experts, public sector organizations and external stakeholders were also consulted during the content development process.

Cognitive testing of the questionnaire content was carried in conjunction with the Questionnaire Design Resource Center based at Statistics Canada in both official languages. For the 2017 iteration, the entire questionnaire was tested through one-on-one interviews with potential respondents that took place in Ottawa, Toronto, Montreal and Vancouver. For the 2019, 2021 and 2022 iterations, the revised content was tested through one-on-one telephone interviews with potential respondents. The resulting comments and analysis of these interviews led to revisions of the questionnaire to make the questions more relevant to respondents and easier to answer.

Sampling

This survey is a census with a cross-sectional design.

The Public Sector Survey on Cyber Security and Cybercrime is designed to obtain complete and accurate data from all organizations from Schedule IV and V of the FAA and federal units from the PSU. No sampling is done.

Data sources

Data collection for this reference period: 2023-01-30 to 2023-03-31

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

Data are collected through an electronic questionnaire. Organizations are initially contacted by telephone during a pre-contact phase to identify an appropriate contact within the enterprise to respond to the survey.

Follow-up because of non-response, inconsistent or missing data is done by phone on a priority basis.

View the Questionnaire(s) and reporting guide(s) .

Error detection

The 2022 Time Use Survey used the Social Survey Processing Environment (SSPE) generalized processing steps and utilities to allow subject matter and survey support staff to specify and run the processing of the survey in a timely fashion with high quality outputs. It used a structured environment to monitor the processing of data ensuring best practices and harmonized business processes were followed.

Edits were performed automatically and manually at various stages of processing at macro and micro levels. They included consistency and flow edits. A series of checks was done to ensure the consistency of survey data. For example, checking the respondent's age against the birth date. Flow edits were used to ensure respondents followed the correct path and fix off-path situations.

Error detection was done through edits programmed into the EQ system, which allowed a valid range of codes for each question and built-in edits, and automatically followed the flow of the questionnaire.

Head office performed the same checks as the EQ system as well as more specific validation of the time diary that was beyond the scope of automated flow and consistency edits. Records with missing or incorrect information were, in a small number of cases, completed, corrected deterministically, or imputed from other information on the questionnaire.

Imputation

Non-response was not permitted for those items required for weighting. Values were imputed in the rare cases where the sex of the respondent was missing. The imputation was based on a detailed examination of the data and the consideration of any useful data such as the age and sex of other household members, and the interviewer's comments.

Information from other questions in the survey was used to impute missing or incorrect information on sleeping, eating, as well as travel and commuting. Several of these questions were added to the 2022 survey based on analysis of data from the 2015 survey, to enhance data quality when issues related to underreporting were present.

Missing information was at times imputed manually based on other responses in the diary. For example, simultaneous activities were used to update primary activities, if appropriate, when the latter was missing.

Estimation

When a probability sample is used, as was the case for the Time Use Survey, the principle behind estimation is that each person selected in the sample represents (in addition to himself/herself) several other persons not in the sample. For example, in a simple random sample of 2% of the population, each person in the sample represents 50 persons in the population (himself/herself and 49 others). The number of persons represented by a given respondent is usually known as the weight or weighting factor.

The Time Use Survey, 2022 is a survey of individuals and the analytic files contain questionnaire responses and associated information from the respondents.

Two weighting factors are available on the microdata file:

WGHT_PER: This is the basic weighting factor for analysis at the person level, i.e. to calculate estimates of the number of persons (non-institutionalized and aged 15 or over) having one or several given characteristics.

WGHT_EPI: This is the basic weighting factor for the analysis at the episode level i.e., to calculate estimates on the number of time an activity is done by the Canadian population. The WGHT_EPI has the same value as the person weight; it does, however, have a different interpretation. It indicates the number of time use episodes that a record on the Episode File represents.

In addition to the estimation weights, bootstrap weights have been created for the purpose of design-based variance estimation.

Estimates based on the survey data are also adjusted (by weighting) so that they are representative of the target population with regard to certain characteristics (each month has independent estimates for various age-sex groups by province). To the extent that the characteristics are correlated with those independent estimates, this adjustment can improve the precision of estimates.

Quality evaluation

Extensive validation and comparisons with distributions of variables collected in other surveys within the GSSP were done. In addition, a cross-survey validation process was undertaken after the final weights were applied to ensure the quality of the information.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects that could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Revisions and seasonal adjustment

This methodology does not apply to this statistical program.

Data accuracy

As the data are based on a sample of persons, they are subject to sampling error. That is, estimates based on a sample will vary from sample to sample, and typically they will be different from the results that would have been obtained from a complete census. More precise estimates of the sampling variability of estimates can be produced with the bootstrap method using bootstrap weights that have been created for this survey. The bootstrap method was used to estimate the sampling variability for all the estimates produced based on the data from the 2022 Time Use Survey.

RESPONSE RATES:
The overall response rate is 30.7%.

NON-SAMPLING ERROR:
Common sources of these errors are imperfect coverage and non-response.

Coverage errors (or imperfect coverage) arise when there are differences between the target population and the surveyed population. Dwellings without telephones and mailable addresses were not collected as we had no way to contact them. To the extent that the excluded population differs from the rest of the target population, the results may be biased. In general, since these exclusions are small, one would expect the biases introduced to be small.

Non-response could occur at several stages in this survey. There were two stages of information collection: at the household level and at the individual level. Some non-response occurred at the household level, and some at the individual level. Survey estimates were adjusted (i.e. weighted) to account for non-response cases.

Other types of non-sampling errors can include response errors and processing errors.

NON-RESPONSE BIAS:
The main method used to reduce nonresponse bias involved a series of adjustments to the survey weights to account for nonresponse as much as possible. Information was extracted from administrative sources and used to model and adjust nonresponse.

COVERAGE ERROR:
The frame for the regular sample was Dwelling Universe File (DUF), a file produced at Statistics Canada. All respondents in the ten provinces were interviewed by telephone or self-completed an electronic questionnaire. Dwellings that were identified as vacant at the time the sampling frame was created were excluded. Dwellings that had neither a mailing address nor an associated telephone number were not collected, as they could not be contacted by any of the survey collection modes. However, the survey estimates were weighted to include persons living in these dwellings.

OTHER NON-SAMPLING ERRORS:
For the 2022 TUS, significant effort was made to minimize bias by using a well-tested questionnaire, a proven methodology, specialized interviewers and strict quality control, and by following up with households that did not initially respond to the survey.

MODE EFFECT:
The 2022 TUS offered an Internet option to almost all survey respondents. This approach to data collection was in recognition of the need to adapt to the changing use of technology and the ever-present demands on Canadians' time. However, there are reasons to believe that the use of an electronic questionnaire might have an impact on the estimations. The impact of the collection mode on the estimates has been analyzed by selecting certain groups of key questions.

Users should refer to the survey user guide and codebook for details on which variables have been identified as being impacted by a strong or mild mode effect.