Survey on the Use of Published Material (SUPM)

Detailed information for 2013

The Survey on the Use of Published Material gathers data on copying from books, magazines, academic journals, and newspapers by federal government employees for work-related purposes.

Data release - October 11, 2013

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

The survey gathers data on copying from books, magazines, academic journals, and newspapers done by federal government employees for work-related purposes. 'Copying' refers to the reproduction, in whole or in part, of published material by photocopying, faxing, printing from an electronic file, scanning, saving to an electronic medium, emailing, and posting to a website. Of interest to the survey is copying from books, magazines, academic journals, and newspapers published by non-governmental publishers or organizations, Canadian and foreign. Material published by governments, other than the Government of Quebec, is excluded.

The survey questionnaire asks federal public servants if they made any copies in the last 12 months and, if so, how many times they made copies in the last 30 days. As well, the questionnaire collects data about the most recent instance of copying, including the reasons for copying.

This information is required by Public Works and Government Services Canada (PWGSC) to prepare for the renewal of a licensing agreement that allows public servants to copy published material.

Subjects

• Government

Data sources and methodology

Target population

The target population consists of public service employees who worked in federal departments and agencies as of February 28, 2013 that fall under Schedule I, I.1, and II of the Financial Administration Act, with the following exceptions:
- non-civilians
- Governor-in-Council appointments
- minister's exempt staff
- employees engaged under a student employment program.

The survey population excluded departments and agencies with less than 400 employees. Also excluded were employees from four departments that refused to participate. This undercoverage was accounted for with an adjustment to the survey weights.

Instrument design

The sponsoring department, PWGSC, drafted a paper questionnaire based on a survey instrument that had been used for a similar earlier survey. Statistics Canada modified this draft in collaboration with PWGSC and then tested it in the Agency's Questionnaire Design Resource Centre.

The electronic format of the questionnaire was designed to follow standard practices and wording, when applicable, in an Internet-based environment. This includes the automatic control of question wording and flows that depended on answers to earlier questions and the use of online edits to check for logical inconsistencies and capture errors such as out-of-range values.

The electronic application for data collection was subjected to extensive testing, including interviews with public servants. Some changes were made to the questionnaire to facilitate understanding of concepts used by the survey.

Sampling

This is a sample survey with a cross-sectional design.

The sampling frame was made up of all in-scope employees that were on the Public Works and Government Services Canada's Incumbent file. Since contact information (email) was not available on the Incumbent file, it had to be collected by Statistics Canada from the departments under the provisions of Article 13 of the Statistics Act.

A sample of approximately 27,000 employees was randomly selected from the file. The sampling unit was the employee.

Data sources

Data collection for this reference period: 2013-05-13 to 2013-06-14

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

Each person in the sample is contacted by e-mail and invited to complete an electronic questionnaire available on the Statistics Canada website.

During collection reminder e-mails are sent on a regular basis to participants in the electronic collection who have not submitted their electronic questionnaire.

View the Questionnaire(s) and reporting guide(s) .

Error detection

Some editing was done directly at the time the electronic questionnaire was completed. Where the information was outside the range (too large or small) of expected values, or inconsistent with the previous entries, the respondent was prompted, through message screens, to verify the information. However, the respondents had the option of bypassing the edits, and of skipping questions if they did not know the answer or refused to answer. Therefore, the data were subjected to further edit processes after they were submitted to head office.

In this survey, there were very few errors in questionnaire flow and few missing answers. As well, there were no instances of incoherent entries.

When a combination of missing answers made a record unusable it had to be dropped from the file. There were six such cases. The records with higher than expected values were reviewed and in one case a follow-up interview was conducted.

Imputation

When the genre of the published document was not specified by the respondent, it was randomly imputed so that the proportions of the reported genres were preserved. Three records were imputed in this way.

Estimation

The SUPM uses a stratified simple random sample design. At the time of selection, an initial design weight was assigned to each person, as the inverse of its probability of selection.

It was observed that non-response did not occur randomly or uniformly within the population since different response rates were obtained for different sub-populations. The use of an appropriate technique is required to correct non-response bias that may be introduced. The chosen technique for SUPM was based on response homogeneous groups (RHG). RHGs were developed with the premise of identifying sample units with similar response probabilities. In other words, it is assumed that persons pertaining to a given RHG are equally likely to respond to the survey in a similar fashion. Many factors, among them gender and age are traditionally known to be factors associated with different non-response patterns. Analyses were completed and the RHGs were identified. The implementation, i.e. the calculation of the weight adjustment, was carried out using Statistics Canada's StatMx software. This approach also ensures the use of the proper variance formula.

Calibration to totals derived from an external file allows the survey to account for the undercoverage due to exclusion of small departments and departmental refusals at the sample design stage of the survey. The totals used for this calibration were derived from the Incumbent file for all in-scope active employees in the target population.

Two variables, an employee's job classification and their annual salary, were correlated with the likelihood of responding to the survey and for those who responded, the likelihood that they were copiers. These variables were used to create the calibration groups.

The final weight consists of multiplying the design weight, the non-response adjustment and the calibration adjustment.

Quality evaluation

Some survey estimates, such as popularity of methods of copying and volume of copies made were compared to the results of a survey on copying of published documents by provincial public service conducted in 2012 by a research organization.

As expected, an employee's job classification was correlated with the likelihood that they made copies of published material.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Given that the Survey on the Use of Published Material did not include questions that could identify respondents, there was no need for disclosure control.

Revisions and seasonal adjustment

This methodology does not apply to this survey.

Data accuracy

Coverage errors and response rate

Public servants from the 43 departments and agencies with less than 400 employees accounted for 1.3% of the target population. Also there were four departments who did not agree to participate; they accounted for 2.4% of the target population. For estimation, the survey weights of the responding units were adjusted to account for this undercoverage.

The sampling frame was made up of all in-scope employees from the Public Works and Government Services Canada's Incumbent file. Since the e-mail address for each employee was not available from the Incumbent file, it had to be collected by Statistics Canada from the departments through Article 13 of the Statistics Act. The files received from the departments were linked to the PWGSC's Incumbent file to create the survey frame. The record linkage was done using the variable "PRI (personal record identifier)". When the variable was not available, the record linkage was done using the variables "Department", "Last name", "First name initial", "Province of work", "Sex", "Position occupational group and level" and "Employee occupational group and level" when provided by the departments. The link rate was 96.6%, which resulted in only 3.4% of the sampled employees with no contact information.

The overall response rate was 44.5%; it varied between departments/agencies - from 26.2% in a small agency and 66.6% in a large agency.


Sampling errors

The basis for measuring the potential size of sampling errors is the standard deviation of the estimates derived from survey results. However, due to the large variety of estimates that can be produced from a survey, the standard deviation of an estimate is usually expressed relative to the estimate to which it pertains. This resulting measure, known as the coefficient of variation (CV) of an estimate, is obtained by dividing the standard deviation of the estimate by the estimate itself and is expressed as a percentage of the estimate.

Statistics Canada commonly uses CV results when analyzing data and urges users producing estimates from the SUPM data files to do so as well.

Non-sampling errors

Over a large number of observations, randomly occurring errors will have little effect on estimates derived from the survey. However, errors occurring systematically will contribute to biases in the survey estimates. Considerable time and effort was devoted to reducing non-sampling errors. Quality assurance measures were implemented at each step of data collection and processing to monitor the quality of the data.

A major source of non-sampling errors in surveys is the effect of non-response on survey results. The extent of non-response varies from partial non-response (failure to answer one or a few questions) to total non-response. Partial non-response to the SUPM was minimal; once the questionnaire was started, it tended to be completed with very little non-response. Total non-response occurred when a person refused to participate in the survey. Total non-response was handled by adjusting the weight of persons who responded to the survey to compensate for those who did not respond.