Public Service Employee Survey (PSES)

Detailed information for 2014

The primary objective of the survey is to obtain the views of federal public service employees about their workforce, workplace and leadership. The survey results highlight where organizations are doing well and identify areas for improvement to help organizations develop informed action plans to address people management issues.

Data release - February 5, 2015

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy

Description

The Public Service Employee Survey (PSES) has been conducted every three years since 1999 to gather employee perspectives on aspects of their workplace, workforce and leadership, providing information about employee engagement, performance management, career development, and fairness and respect in the workplace. The survey results provide essential information for the Management Accountability Framework, and inform policy related to values and ethics, official languages, staffing, training, and other key people management areas.

As of 2014, survey results can be compared over three survey cycles to reveal trends across the public service or within organizations. Some questions of the PSES allow for benchmarking with the results of employee surveys of other governments, including provincial/territorial and international comparisons.

The survey results inform managers and employees about strengths and areas for improvement at all levels within an organization. The results contribute to the understanding of people management issues, leading to action plans that may positively impact the workplace. The results also serve as a platform to initiate and maintain dialogue about key people management issues.

Subjects

• Government
• Industries
• Labour

Data sources and methodology

Target population

The survey targets active employees of organizations in the core public administration and of participating separate agencies listed in Schedules I, IV and V of the Financial Administration Act. Indeterminate, term, seasonal, casual and student employees, as well as Governor in Council appointees are eligible to participate.

Instrument design

The content of the 1999 survey was developed by an interdepartmental committee, led by the Treasury Board of Canada Secretariat, with the support of Statistics Canada. In 2002, the survey was modified extensively, retaining 39 questions from the 1999 version. The 2005 version was a duplicate of the 2002 PSES. In 2008, the survey underwent a major revision, including changes to the response scale for the majority of questions, which precluded comparisons with results from previous survey cycles. Both the 2011 and 2014 questionnaires have evolved to address current issues and allow for benchmarking with other government employee surveys. The 2014 survey content was developed through extensive consultation with departments and agencies, central agencies, bargaining agents, Human Resources policy groups, functional communities, and employment equity group committees.

The 2014 questionnaire contained 106 questions: 17 new questions, 12 modified questions, and 76 questions repeated from the 2011 survey (55 of the repeated questions were also used in the 2008 survey). To test the content of the 2014 questionnaire, focus group sessions were held in the National Capital, Winnipeg and Montréal. Participants were from various departments and agencies, and various occupational groups and levels.

As well, participating departments and agencies were given an opportunity to add up to five supplementary questions to be administered to their employees and, 13 did. The supplements were also tested in a series of focus groups held in each of these departments.

The 2014 questionnaire was formatted as an electronic survey (to be completed online), as a paper survey and in three alternate formats (large print, Braille and audio CD).

Sampling

This survey is a census with a cross-sectional design.

Data are collected for all units of the target population; therefore, no sampling is done.

The PSES sample frame is built from lists of employees provided by human resource services from participating organizations. In doing this, they have to provide a list of fields requested by Statistics Canada to conduct the survey and to include employees meeting the in-scope population criteria. These lists were then verified, cleaned and combined together by Statistics Canada to form the final survey frame. This sampling unit is person.

A total of 93 departments and agencies participated in the 2014 survey.

Data sources

Data collection for this reference period: 2014-08-25 to 2014-10-31

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

The Public Service Employee Survey was administered to active employees in organizations in the federal public service, for which Treasury Board is the employer, as well as in participating separate agencies. The survey was a census with voluntary participation. The collection was primarily done using an electronic questionnaire. Each department and agency was responsible for providing a complete list of email addresses for their department. Email invitations were sent to each employee with a valid email address containing a unique password to access the survey questionnaire. Reminders were sent on a weekly basis to those who have not responded to the survey. As soon as the respondent submitted their completed questionnaire, the data were transferred through Statistics Canada's internal network and then decrypted for processing. Respondents had the possibility to save their partially completed questionnaire and finish it later.

The 2014 electronic collection ran from August 25 to October 3, 2014. The survey closing date was extended by one week, from September 26 to October 3.

Employees who did not have email addresses or access to the Internet received paper questionnaires, which were distributed through the human resources service of their department or agency. Paper questionnaires were returned directly to Statistics Canada in a postage-paid return envelope. The closing date for acceptance of paper questionnaires was October 31, 2014.

Alternate format questionnaires were made available upon request.

View the Questionnaire(s) and reporting guide(s) .

Error detection

The data capture of the 3,280 paper questionnaires received was carried out between September 2014 and October 2014. The data were captured using imaging and automated data entry technology. A small proportion of questionnaires, those that could not be read by the optical scanners, were captured using heads-down keying by experienced operators. Questionable zones method with standard quality control measures were used to verify the error rate of the capture operations. For the Public Service Employee Survey, based on the quality control sample that was selected, it was determined that the overall data capture error rate did not exceed 0.5%.

All 182,165 paper and electronic questionnaires received were processed the same way. The data were processed by applying edit rules to identify missing, invalid or inconsistent data. Each question was examined to verify the presence of a valid code. If none was present then a "Not stated" response code of "9" was assigned. For example, an edit rule was also applied that examined the flow of data from Question xx to Question xy. Depending on the response to Question xx, superfluous data that did not respect the flow of data were eliminated and coded as a "Valid Skip".

Imputation

This methodology does not apply.

Estimation

The weight for the Public Service Employee Survey is calculated so that the respondent and population distributions have the same overall distribution with respect to the department/agency and the aggregate occupational group.

Simply put, if 20% of the employees in a department or agency are in a particular aggregate occupational group, then the weight ensures that this aggregate occupational group represents 20% of the number of respondents when tabulating the data. In other words, the weight compensates for the over and under representation of aggregate occupational groups within each federal department/agency. For aggregate occupational groups that were over represented within the department/agency, the weights are smaller than one. For aggregate occupational groups that were under represented within the department/agency, the weights are greater than one. That is, if the weight is larger than one then each person represents, besides himself or herself, other persons who did not respond. This weight indicates that the aggregate occupational group was under represented within the department/agency. For example, if the weight is 2, each person represents 2 persons in the population.

The weighting step calculates this number for each record. This weight must be used to derive estimates from the microdata file.

For example, if the number of respondents who "Strongly agree" with the statement "I am proud of the work I do" is to be calculated, it is done by selecting the records for those respondents (Q15 = 1) and summing the weights.

Up to and including 2008 the weights calculated for the PSES were not designed to inflate the respondents so that they represented the population. Non-response weighting adjustments were made to reduce non-response bias, but the weights add up to the number of respondents within each department/agency, not the population size. Therefore when releasing demographic estimates, no statements to that effect can be made.

Note that no adjustment for non-response in small departments and agencies was done due to the small number of employees within the occupational groups in these departments and agencies.

Quality evaluation

While rigorous quality assurance mechanisms are applied across all steps of the statistical process, validation and scrutiny of the data by statisticians are the ultimate quality checks prior to dissemination. Two validation measures were implemented. They include: 1) analysis of changes over time and 2) verification of estimates through cross-tabulations.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects that could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

In order to prevent any data disclosure, confidentiality analysis is done using the Statistics Canada Generalized Disclosure Control System (G-Confid). G-Confid is used for primary suppression (direct disclosure) as well as for secondary suppression (residual disclosure). Direct disclosure occurs when the value in a tabulation cell is composed of or dominated by few enterprises while residual disclosure occurs when confidential information can be derived indirectly by piecing together information from different sources or data series.

Revisions and seasonal adjustment

This methodology does not apply to this survey.

Data accuracy

The overall response rate for the 2014 Public Service Employee Survey was 71.4%.

The Public Service Employee Survey is a census and therefore, there is no error due to sampling. However, the survey is subject to non-sampling errors such as non-response or other non-sampling errors that may occur at almost every phase of a survey operation. Respondents may make errors in answering questions, the answers may be incorrectly captured and errors may be introduced in the processing and tabulation of the data.

Quality assurance and control methods were implemented according to Statistics Canada's standard practices at each step of the PSES 2014 data collection and processing cycle to monitor the quality of the data. These measures included focus group testing to detect problems of questionnaire design or misunderstanding of instructions, and using edit rules designed to detect missing, invalid or inconsistent data.

Quality control was also used to measure the error rate for a sample of the data captured by the operators and automatically. Errors found in the sample were corrected. For the PSES 2014, based on the quality control sample that was selected, it was determined that the overall data capture error rate did not exceed 0.5%.
Total non-response can be a major source of non-sampling error in many surveys, depending on the degree to which respondents and non-respondents differ with respect to the characteristics of interest. Total non-response occurred when an eligible employee did not participate in the survey or returned a completely blank questionnaire. The percentage of respondents to the number of employees in the public service population for 2014 was 71.4%. The Treasury Board Secretariat Incumbent System file was used to represent the public service population.

Nonresponse adjustments were calculated separately for each department or agency from which 50 or more employees responded to the 2014 PSES. As in 2011, the nonresponse groups were defined by two-letter occupational group where possible, or collapsed within the aggregate occupation group.