Public Service Employee Survey (PSES)

Status:
Active
Frequency:
Occasional
Record number:
4438

The primary objective of the survey is to obtain the views of all employees of the federal Public Service about their workplaces. The information will allow managers and employees to initiate concrete actions in their own department, and where warranted, across the Public Service.

Detailed information for 2011

Data release - February 2, 2012

Description

The effects of Program Review, government restructuring, increased workload and rapid technological advances have greatly affected federal Public Service employees. Recent studies and reports on specific segments of the federal Public Service had shown that low morale was prevalent among executives and knowledge workers and that many employees felt that workplace conditions were not conducive to confidence in management, job satisfaction and career advancement. Much additional information was required in order to further evaluate these findings and determine how the present workplace structure could be improved to meet the challenges facing it at the turn of the new millennium.

In 1997, the Clerk of the Privy Council introduced the idea of a voluntary survey of all federal Public Service employees. The Treasury Board Secretariat (TBS) was asked to implement the project. The TBS worked in consultation with other key federal departments to develop a national survey that would gather information from all employees through a common questionnaire. As part of the project, Statistics Canada was asked to participate in the development and to collect and process the data.

The survey results will be used to develop actions at the level of the department, sector or branch and ultimately at the work unit level. The results would also serve as input to the future corporate management agenda. The survey will provide a baseline against which future progress in renewing the workplace can be measured.

Subjects

  • Government
  • Industries
  • Labour

Data sources and methodology

Target population

The target population is all employees of the federal Public Service employed under Schedule I, Part I of the Public Service Staff Relations Act (PSSRA 1-1) and employees of separate agencies that agreed to participate.

Instrument design

The questionnaire content was developed by an Interdepartmental Committee comprising representatives from small, medium and large departments/agencies, as well as representation from the Small Agencies Group, Statistics Canada, central agencies, bargaining agents and an external advisor. The questions for this survey were chosen based on their usefulness to employees, managers and bargaining agents in helping to identify problems and provide concrete solutions to improve the work environment. The 2002 and 2005 Public Service Employee Surveys were follow-up surveys to the 1999 PSES. In order to ensure comparability between the 1999 and 2002 surveys, key questions from the 1999 PSES remained identical in the 2002 PSES, while minor changes were made to some other questions. New questions were added in place of certain questions from the 1999 PSES in order to explore new themes and provide additional information on issues identified in the first survey. There was no change between the 2002 and 2005 survey questionnaire.

In 2008, the 2005 questionnaire was used as the basis for the survey. New questions were added to construct an employee engagement model that will be used to evaluate each organization. As well, the scale of the response category was increased from 4 to 5 to include a neutral category.

In 2008, focus groups were held in the National Capital Region and included employees at various groups and levels as well as English and French focus groups. Comments from the focus groups were integrated into the questionnaire and a final layout was decided. Participating departments and agencies were also invited to provide comments on the draft questionnaire. All Survey Champions (departments and agencies) were invited to briefings on the questionnaire and project plan. The Minister responsible for the Treasury Board of Canada Secretariat approved the delivery of the second survey.

In 2011 the survey content was modified to refine the employee engagement model, with 79 questions based on previous versions of the survey and 13 new questions that were developed and qualitatively tested in 2011. The survey also contained six questions that were developed and qualitatively tested in 2009. Focus groups were held in the National Capital Region and included employees at various groups and levels as well as English and French focus groups. Comments from the focus groups were integrated into the questionnaire and a final layout was decided. Participating departments and agencies were also invited to provide comments on the draft questionnaire. All Survey Champions (departments and agencies) were invited to briefings on the questionnaire and project plan.

As the department code was essential for the analysis of the data, it was decided that each department and agency would receive their own questionnaire with their department code on the front page. Eighty-nine departments and agencies participated in the survey of which twenty-seven small organizations were regrouped as one single department. Having the organizational unit lists coded with the same department number made it easier to ensure that the proper list of organizational units would accompany the right questionnaire. The list of organizational units and a postage-paid return envelope were included with the questionnaire in the envelope.

Sampling

This survey is a census with a cross-sectional design.

Data are collected for all units of the target population; therefore, no sampling is done.

Data sources

Data collection for this reference period: 2011-08-29 to 2011-10-07

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

The Public Service Employee Survey was administered to all employees in the Public Service for which Treasury Board is the employer as well as employees of participating separate agencies. The survey was a census. The collection was done using an Electronic questionnaire. Each department and agency was responsible to provide a complete list of email addresses for their department. Email invitations were sent to each employee containing a unique password to access the survey questionnaire. Reminders were sent on a weekly basis to people not having responded to the survey. As soon as the respondent submitted their completed questionnaire, the data was transferred on STC internal network and then decrypted for processing. The respondent had the possibility to save their partially completed questionnaire and finish it later. Electronic collection was closed as of October 7, 2011.

Employees not having access to the internet were to receive a paper questionnaire to complete. The survey was anonymous; that is, the respondent's name or other identification was not required on the questionnaire.

If paper questionnaire was necessary, the department was responsible for distributing the questionnaires to these employees. Once completed, the paper questionnaire was returned directly to Statistics Canada in a postage-paid return envelope. Statistics Canada accepted completed questionnaires for two weeks following the established survey period. The closing date for acceptance of paper questionnaires was October 21, 2011.

View the Questionnaire(s) and reporting guide(s) .

Error detection

The data capture of the 6030 paper questionnaires received was done between September 2011 and October 2011. The data were captured using imaging and automated data entry technology. A small proportion of questionnaires, those that could not be read by the optical scanners, were captured using heads-down keying by experienced operators. Questionable zones method with standard quality control measures were used to verify the error rate of the capture operations. For the Public Service Employee Survey, based on the quality control sample that was selected, it was determined that the overall data capture error rate did not exceed 0.5%.

All 201,430 paper and electronic questionnaires received were processed the same way. The data were processed by applying edit rules to identify missing, invalid or inconsistent data. Each question was examined to verify the presence of a valid code. If none was present then a "Not stated" response code of "9" was assigned. For example, an edit rule was also applied that examined the flow of data from Question 54 to Question 55. Depending on the response to Question 54, superfluous data that did not respect the flow of data were eliminated and coded as "Valid Skip".

As well, data inconsistencies were corrected. Some verification was done to match the province of work and the work unit. An edit was applied in the National Capital Region (NCR) where respondents coded their province of work as being Ontario or Quebec instead of the separate NCR code. In other cases, when regions were identified separately, the department or agency was consulted and when applicable, personnel in the Regions were recoded to a regional unit.

Estimation

The weight calculated for the Public Service Employee Survey can be thought of as re-weighting the respondents so that the respondent and population distributions have the same overall distribution with respect to the department/agency and the aggregate occupational group.

Simply put, if 20% of the employees in a department or agency are in a particular aggregate occupational group, then the weight ensures that this aggregate occupational group represents 20% of the number of respondents when tabulating the data. In other words, the weight compensates for the over and under representation of aggregate occupational groups within each federal department/agency. For aggregate occupational groups that were over represented within the department/agency, the weights are smaller than one. For aggregate occupational groups that were under represented within the department/agency, the weights are greater than one. That is, if the weight is larger than one then each person represents, besides himself or herself, other persons who did not respond. This weight indicates that the aggregate occupational group was under represented within the department/agency. For example, if the weight is 2, each person represents 2 persons in the population.

The weighting step calculates this number for each record. This weight must be used to derive estimates from the microdata file.

For example, if the number of respondents who "Strongly agree" with the statement "I am proud of the work carried out in my work unit" is to be calculated, it is done by selecting the records for those respondents (Q33 = 1) and summing the weights.

Up to and including 2008 the weights calculated for the PSES were not designed to inflate the respondents so that they represent the population. Non-response weighting adjustments were made to reduce non-response bias, but the weights add up to the number of respondents within each department/agency, not the population size. Therefore when releasing demographic estimates, no statements to that effect can be made.

Note that no adjustment for non-response in small departments and agencies was done due to the small number of employees within the occupational groups in these departments and agencies.

The weighting method was modified in 2011. Similar to 2008, the weighting classes were defined by occupational group or category. Unlike previous occasions, the 2011 method ensures that weighted estimates represent the population. Compared to 2008, the 2011 method results in higher weights to less populated occupational groups and slightly lower weights to more populated occupational groups. The 2011 method also ensures that each response has a weight of at least one. Unlike the 2008 weights, the 2011 weights can be used with variance estimation, treating the set of responses as comprising a sample and treating the nonresponse as missing completely at random.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Data accuracy

While considerable effort is made to ensure high standards throughout all stages of collection and processing, the resulting estimates are inevitably subject to a certain degree of error. These errors can be broken down into two major types: non-sampling and sampling.

This is a census and therefore, there is no error due to sampling. However, the survey is subject to non-sampling errors such as non-response or other non-sampling errors that may occur at almost every phase of a survey operation. Respondents may make errors in answering questions, the answers may be incorrectly captured and errors may be introduced in the processing and tabulation of the data.

Quality assurance and control methods were implemented according to Statistics Canada's standard practices at each step of the data collection and processing cycle to monitor the quality of the data. These measures included focus group testing to detect problems of questionnaire design or misunderstanding of instructions, and using edit rules designed to detect missing, invalid or inconsistent data.

Non-response is an important source of non-sampling error. The target population for 2011 consisted of 278,906 individuals. The overall response rate for the 2011 Public Service Employee Survey was 72.2%.