Research and Development of Canadian Private Non-profit Organizations (RDNP)

Detailed information for 2014

The Research and Development of Canadian Private Non-Profit Organizations survey produces useful statistical information to monitor science and technology activities in Canada and to support the development of science and technology policy.

Data release - August 15, 2016

• Questionnaire(s) and reporting guide(s)
• Description
• Data sources and methodology
• Data accuracy
• Documentation

Description

The Research and Development of Canadian Private Non-Profit Organizations (RDNP) is a cross economy survey of private non-profit organizations in Canada that perform or fund research and development (R&D). The survey targets private non-profit organizations that are not part of the government, higher education or industrial sectors.

The concepts and definitions employed in the collection and dissemination of research and development (R&D) data are provided in the Frascati Manual 2015, published by the Organisation for Economic Cooperation and Development (OECD, 2015). According to this definition:

"Research and experimental development (R&D) comprise creative and systematic work undertaken in order to increase the stock of knowledge - including knowledge of humankind, culture and society - and to devise new applications of available knowledge."

The RDNP collects in-house R&D expenditures and personnel, payments to others to perform R&D and payments and receipts for technology.

In-house R&D expenditures include current costs (comprised of wages and salaries of permanent, temporary and casual employees; services to support R&D; R&D materials; and all other current costs) and capital costs (comprised of software; land; buildings and structures; and equipment, machinery and all other capital costs). In-house R&D expenditures are characterized by their geographic distribution (provinces and territories), sources of funds (originating sector inside or outside Canada), fields of research and development and nature of R&D activity (basic research, applied research and experimental development).

In-house R&D personnel include researchers and research managers; R&D technical, administrative and support staff; and other R&D occupations and these data are available by geographic distribution (provinces and territories).

Payments to others to perform R&D comprise payments made to other organizations to perform R&D and may be directed to other organizations (comprising companies; private non-profit organizations; industrial research institutes or organizations; hospitals; universities; federal government departments or agencies; provincial government departments, ministries and agencies; provincial research organizations or other organizations or individuals) inside or outside Canada.

Technology payments include payments made or received for patents, copyrights, trademarks, industrial design, integrated circuit topography designs, original software, packaged off-the-shelf software, databases with a useful life exceeding one year, other technical assistance, industrial processes and know-how. Technology payments can be made to, or received from, affiliated organizations or other organizations within or outside Canada.

Data from the Research and Development of Canadian Private Non-Profit Organizations (RDNP) provide total estimates of in-house R&D expenditures by the private non-profit sector and contribute to gross domestic expenditures on research and development (GERD). These data serve many users from: federal and provincial government science policy analysts who develop and monitor programs related to science and technology activities in the Canadian economy to international organizations such as the Organisation for Economic Co-operation and Development (OECD) and the United Nations Educational, Scientific and Cultural Organization (UNESCO). University researchers, research councils, business enterprises, research institutes and associations, science journal writers, the general public and the media are all users of R&D data.

The results are published in The Daily and on the Common Output Data Repository (CODR). Private non-profit R&D data are combined with data from other R&D performing sectors: Research and Development in Canadian Industry (record no. 4201), Federal Science Expenditures and Personnel, Activities in the Social Sciences and Natural Sciences (record no. 4212), higher education (record no. 5109), and Provincial.

Statistical activity

The survey is administered as part of the Integrated Business Statistics Program (IBSP). The IBSP program has been designed to integrate approximately 200 separate business surveys into a single master survey program. The survey instrument conforms to the common look, structure and content for business surveys in this integrated program.

Reference period: The fiscal year for fiscal year end date between April 1, RY and March 31, RY+1

Subjects

• Research and development
• Science and technology

Data sources and methodology

Target population

The RDNP target population comprises all private non-profit organizations that perform and/or fund research and development (R&D) within the reference period, such as voluntary health organizations, private philanthropic foundations, associations and societies, and research institutes. These not-for-profit organizations serve the public interest by supporting activities related to issues of public welfare (such as health, education, and the environment). The survey excludes those private non-profit organizations that are part of the government, higher education, or business enterprise sectors. These units are identified by NAICS codes (NAICS 61131 for universities and NAICS 91 for public administration), and by non-profit flags on Statistics Canada's Business Register. All units on the RDNP survey frame are reviewed annually by subject matter officers to ensure that units falling under the control and direction of business enterprise, higher education or government sectors are excluded from collection, in order to prevent data duplication.

Instrument design

The Research and Development of Canadian Private Non-Profit Organizations (RDNP) introduced an electronic questionnaire (E-Q) for reference year 2014. The E-Q content was developed to conform to international standards for research and development concepts (OECD, Frascati Manual 2015). The first round of questionnaire content testing occurred in March, 2014 in English in Ottawa, Toronto and Montreal and in French in Gatineau and Montreal. Testing private non-profit respondents' understanding of specific concepts, questions, terminology including the appropriateness of response categories and the availability of the requested information were the main purposes of this test. Based on respondent feedback, the questionnaire content was revised and a second round of content testing occurred in April, 2015 in the E-Q format. These tests were conducted in English in Toronto and in French in Montreal. The E-Q functionality testing confirmed that respondents were able to navigate through the questionnaire contents, response, help screens and edits with ease while providing the requested information. The content testing indicated that the questionnaire content specific to research and development functioned successfully. However, respondents indicated concepts related to the reference year were unclear. A final test, conducted in May and June, 2015 in English and French in Ottawa and French in Montreal occurred to confirm that the revised question wording on reference year met respondents' requirements. The E-Q applications used for the RDNP adhered to all standards and guidelines in place at time of development.

Sampling

This survey is a census with a cross-sectional design.

The survey comprises a census of all statistical enterprises taken from a survey frame of private non-profit organizations known or believed to be performing or funding R&D in the reference period. There are about 200 active units on the survey frame. All active units on the survey frame for the reference period are selected to receive the electronic questionnaire. The survey frame or population comprises private non-profit organizations with a high degree of variability in organization size determined by employment, annual financing available for R&D, business strategies towards performing R&D in-house or granting or contracting out R&D to other organizations, geographic distribution of R&D activities as well as operating control of the organizations. For these reasons, a census was determined to be the most suitable survey design.

The survey frame is updated and reviewed annually. It is based upon compilation of private non-profit organizations that have reported R&D expenditures for the current period or within two years prior to the reference period by questionnaire or otherwise reported R&D expenditures to Statistics Canada's Business Activity, Expenditures and Output Survey (BAEO), Capital and Repair Expenditures Survey (CAPEX), Federal Science Expenditures and Personnel (FSEP) survey, other ad hoc surveys or indicated R&D expenditures through media and annual reports.

Administrative data used for frame maintenance and frame building are obtained from Canada Revenue Agency forms (T3010 "Registered Charity Information Return" and T1044 "Non-Profit Organization (NPO) Information Return). The text fields describing the activities of the organization are searched for key words in English and French to identify organizations which may be in scope for the RDNP survey. These organizations are then contacted to determine if they should be included in the survey frame. Employment size is calculated from "Current Source Deduction Remittance Voucher" (PD7) data on Statistics Canada's Business Register. All units on the survey frame are verified to ensure that they are not included in the government, higher education or business enterprise sectors and that their operating structure has not changed from the prior year.

Data sources

Data collection for this reference period: 2015-11-12 to 2016-02-15

Responding to this survey is mandatory.

Data are collected directly from survey respondents.

Electronic questionnaire with non-response follow-up and failed edit follow-up

View the Questionnaire(s) and reporting guide(s) .

Error detection

Error detection is an integral part of both collection and data processing activities. Automated edits are applied to data records during collection to identify reporting and capture errors. These edits identify potential errors based on year-over-year changes in key variables, totals, and ratios that exceed tolerance thresholds, as well as identify problems in the consistency of collected data (e.g. a total variable does not equal the sum of its parts). During data processing, other edits are used to automatically detect errors or inconsistencies that remain in the data following collection. These edits include value edits (e.g. Value > 0, Value > -500, Value = 0), linear equality edits (e.g. Value1 + Value2 = TotalValue), linear inequality edits (e.g. Value1= Value2), and equivalency edits (e.g. Value1 = Value2). When errors are found, they can be corrected using the failed edit follow up process during collection or via imputation. Extreme values are also flagged as outliers, using automated methods based on the distribution of the collected information. Following their detection, these values are reviewed in order to assess their reliability. Manual review of other units may lead to additional outliers identified. These outliers are excluded from use in the calculation of ratios and trends used for imputation, and during donor imputation. In general, every effort is made to minimize the non-sampling errors of omission, duplication, misclassification, reporting, and processing.

Imputation

When non-response occurs, when respondents do not completely answer the questionnaire, or when reported data are considered incorrect during the error detection steps, imputation is used to fill in the missing information and modify the incorrect information. Many methods of imputation may be used to complete a questionnaire, including manual changes made by an analyst. The automated, statistical techniques used to impute the missing data include:
• deterministic imputation (for example adding components to create a total),
• replacement using previously reported anticipated values for the current period values (the survey asks for reference year (RY) and RY+1 and RY+2 values for key variables),
• replacement using historical data (with a trend calculated, when appropriate),
• replacement using auxiliary information available from other sources,
• replacement based on known data relationships for the sample unit, and
• replacement using data from a similar unit in the sample (known as donor imputation).
For the private sector research and development surveys, the key question on expenditures for in-house R&D is verified or imputed first and then these values are used as anchors in subsequent steps to impute other, related, variables.
Imputation generates a complete and coherent micro data file that covers all survey variables.

Estimation

The RDNP population is small as R&D is a rare activity. The survey is therefore conducted as a census. The frame is not stratified for sampling purposes and respondents have a weight of 1. The survey population is stratified for edit and imputation purposes. The strata are: fields of R&D (comprising three groups: 1) medical sciences, 2) natural sciences other than medical sciences and 3) social sciences and humanities) and size (comprising three groups: 1) small (< 10), 2) medium (>9 and< 50) and 3) large (>50) based on total employment).

Quality evaluation

Prior to the data release, combined survey results are analyzed for comparability. This analysis includes a detailed review of:
• individual responses (especially for the largest organizations),
• general public funding announcements and initiatives,
• coherence with results from other R&D surveys,
• prior period intentions, and
• information from other external sources (e.g. annual reports, news articles).
The survey estimates are also analyzed with trends observed in data from previous collection cycles and media reports.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects that could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

In order to prevent any data disclosure, confidentiality analysis is done using the Statistics Canada Generalized Disclosure Control System (G-Confid). G-Confid is used for primary suppression (direct disclosure) as well as for secondary suppression (residual disclosure). Direct disclosure occurs when the value in a tabulation cell is composed of or dominated by few enterprises while residual disclosure occurs when confidential information can be derived indirectly by piecing together information from different sources or data series.

Data accuracy

There are two types of errors to which survey data can be subject: sampling errors and non-sampling errors. Since this survey is a census, this survey is not subject to sampling error. Non-sampling error is not related to sampling and may occur for various reasons during the collection and processing of data.
Non-sampling errors include:
• Non-response (both total and partial)
• Under or over-coverage of the population
• Differences in the interpretations of questions and mistakes in reporting
• Coding and processing errors
To the maximum extent possible, these errors are minimized through careful design of the survey questionnaire, verification of the survey data, and follow-up with respondents when needed to maximize response rates.
Imputation rates can be estimated to generate a quality rating code. The imputation rate is calculated based on the contribution of imputed values to the total estimate. The quality indicator code uses letters that ranges from A to F where A means the data are of excellent quality and estimates with a quality of F are too unreliable to be published. These quality rating codes should always be taken into consideration when using the estimates.
Another indicator of quality is the survey response rate.

Documentation

• Changes to the survey for RY2014
  • Format: Changes to the survey for RY2014 - ARCHIVED - HTML[ARCHIVED - HTML]