Farm Environmental Management Survey (FEMS)

Detailed information for 2011

Status:

Active

Frequency:

Every 5 years

Record number:

5044

This survey, focusing on both livestock and crop operations, will allow the establishment of base lines and development of updates for an expanded set of agri-environmental indicators, and generate the information to design effective and well targeted policy and program responses.

Data release - February 7, 2013

Description

The FEMS contributes to Agriculture and Agri-Food Canada's work on measuring environmental performance in the agricultural sector. The information generated from this survey is needed to support the industry's environmental initiatives, to address federal and provincial policy needs and to guide sustainable development actions in Canada's agriculture sector. This voluntary survey, focusing on both livestock and crop operations will allow the establishment of base lines and development of updates for an expanded set of agri-environmental indicators. These indicators are needed to: determine the present status of farm environmental management across Canada; identify areas that are most in need of environmental management movements; and generate the information to design effective and well targeted policy and program responses. This survey is conducted in conjunction with Agriculture and Agri-Food Canada, to ensure that agriculture programs reflect the changing way resources are being managed on today's farms.

Reference period: Calendar year

Subjects

  • Agriculture
  • Environment
  • Environmental protection
  • Land use and environmental practices

Data sources and methodology

Target population

The conceptual universe is made up of all active farms in the Agriculture Division's Farm Register. The following types of farms were excluded:
. Active farms with agricultural sales in 2010 of less than $10,000;
. Institutional farms (prisons, research stations, colleges);
. Farms located on Indian reserves;
. Farms for whom more than 50% of the gross income in 2010 came from sales of greenhouse, sod and nursery products;
. Farms included on the 2012 Greenhouse, Sod and Nursery Survey frame
. Farms without livestock inventory or crop area at the time of the 2011 Census of Agriculture;
. Farms located in the Yukon, the Northwest Territories and Nunavut;
. Farms part of the Large Agricultural Operation Statistics (LAOS) program.

Instrument design

The Farm Environmental Management Survey questionnaires were designed by a project team made up of Statistics Canada and Agriculture and Agri-Food Canada employees and provincial experts assigned to the project. Questionnaire design specialists were consulted within Statistics Canada. In November 2010, 56 one-on-one in-depth interviews with farm producers in five regions were used to test draft versions of the questionnaires: Kelowna, British Columbia, Brandon, Manitoba, Welland, Ontario, Trois-Rivières, Quebec and Wolfville, Nova Scotia. Participants represented various types of agricultural operations. The questionnaires were then revised based on results of the questionnaire testing, questionnaire design specialist recommendations and a second round of consultation with Agri-Food Canada employees and provincial focal points or experts.

Sampling

This is a sample survey with a cross-sectional design.

There was a requirement to report the survey results for 27 sub-provincial regions. The combination of 12 ecological regions (described below) and 10 provinces produced 27 different sub-provincial regions defined by similar agronomic, climatic and soil attributes in the Canadian eco-stratification system.

The 12 regions by which the survey frame was stratified are as follows:

1) Atlantic Maritime ecozone
2) St. Lawrence Lowlands ecoregion
3) Manitoulin- Lake Simcoe - Frontenac ecoregions
4) Lake Erie Lowland ecoregion
5) Boreal Shield ecozone
6) Brown Soil Zone (ecoregions Mixed Grassland and Cypress Upland)
7) Dark Brown Soil Zone (ecoregions Moist Mixed Grassland and Fescue Grassland)
8) Black Soil Zone (ecoregions Aspen Parkland and Southwest Manitoba Uplands)
9) Lake Manitoba Plain ecoregion
10) Boreal Plains ecozone
11) Montane Cordillera ecozone
12) Pacific Maritime ecozone

Stratification
The first step was to determine whether each farm in the sampling frame was a candidate for the Crop module (or questionnaire), for the Livestock module, or for both (a Mixed farm). The farms were assigned to one of the three groups (Crop, Livestock or Mixed) according to their contribution to cropland and to livestock units in their province. The population was then stratified according to the province and ecological region. Distributions of the gross farm income and the crop type (wheat, grain, oilseed, potatoes, fruits, vegetables, hay, other field crop and mixed crop) and/or livestock type (cattle, hogs, poultry, mixed livestock) included on the frame were compared to distributions of the final sample, to ensure that each type of farm was well-represented.

Sample allocation
Based on the available budget, the expected response rate and a targeted level of precision, the total sample size was set at 20,000 farms, roughly 10,000 for each module. Within a given stratum of the Crop module, the sample was split between farms with only crops and Mixed farms, in such a way that their distribution was the same as in the target population. The same strategy was used for the Livestock module.

Sample Selection
Farms were randomly selected within each stratum. The sampling strategy ensured that a farm could not be selected for both the Crop and Livestock modules.

Two main sources of frame under-coverage existed: new farms that started their activities after the 2011 Census of Agriculture and farms with sales less than the $10,000 threshold in 2010 (the reference year for the 2011 Census of Agriculture) but which had surpassed the threshold since then.

Data sources

Data collection for this reference period: 2012-02-03 to 2012-03-30

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

As suggested by participants during questionnaire testing, February-March was chosen for data collection because it had the least impact on farming operations (before spring planting). The survey also tied in well with the 2011 Census of Agriculture completed in the summer of 2011. A Computer Assisted Telephone Interview data collection technique was developed as the data collection method for this survey.

View the Questionnaire(s) and reporting guide(s).

Error detection

Data were verified using the edits in the capture software. These edits verify that all mandatory cells have been filled in, that certain values lie within acceptable ranges, and that totals equal the sum of their components. If an edit was triggered, collection officers asked follow-up questions to the respondent in order to resolve the edit.

Further data checking was performed by subject matter officers who reviewed the survey data that were identified as outliers and compared them to data from the 2011 Census of Agriculture to determine their validity.

Imputation

No statistical imputation of the survey data was performed to address non-response. Total non-response (that is, when mandatory questions are left unanswered) was dealt with by adjusting the weights assigned to the responding units, such that one responding unit might also represent other non-responding units with similar characteristics (that is, province, ecozone or ecoregion, farm type).

Estimation

The collected survey data were weighted in order to produce unbiased estimates which were representative of the population. These weights reflected the sample design and the non-response observed during collection. Estimates were produced at the province and ecoregion levels where there was sufficient response to produce reliable statistical aggregated data.

The estimates were produced using Statistics Canada's Generalized Estimation System.

Quality evaluation

The data collected from the survey were compared to the Census of Agriculture data. Estimates were also compared to other comparable published estimates and analyzed by subject matter experts. The estimates at the provincial level were also validated by the provincial focal points who were involved in the design of the survey questionnaire and who are knowledgeable about the subject matter on a regional basis. Finally, the results of the survey were also compared, whenever possible, to the previous results of the Farm Environmental Management Survey.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

To reduce response burden and to ensure more uniform statistics, Statistics Canada has entered into an agreement under section 12 of the Statistics Act with Agriculture and Agri-Food Canada, the Ministère de l'Agriculture, des Pêcheries et de l'Alimentation du Québec, the Institut de la statistique du Québec, the Ontario Ministry of Agriculture, Food and Rural Affairs and the Alberta Agriculture and Food Ministry. Under Section 12, Statistics Canada will not share any name, address or other identifying respondent information. The information is required to be kept confidential and used only for statistical and research purposes. These various data sharing partners will only have access to respondents within their respective province who agree to share survey information.

Revisions and seasonal adjustment

This methodology type does not apply to this statistical program.

Data accuracy

While non-sampling errors are difficult to quantify, sampling errors can be estimated from the sample itself using the standard error (SE) of estimated values also referred to as an absolute sampling error. For level estimates (e.g. totals and averages), a statistical measure called the coefficient of variation (CV) is normally used. The CV, defined as the standard error divided by the survey estimate, is a measure of precision in relative terms and is expressed as a percentage.

For level estimates, the CV is the appropriate measure of the sampling error. For proportions, however, an absolute sampling error -- such as the SE itself -- is preferred. In the case of the Farm Environmental Management Survey, most of the estimates deal with proportions.

The SE (which is a function of the population size, the sample size, and the estimate), along with the confidence level, can be used to calculate the margin of error. This measure is straightforward to interpret, since it is on the same scale as the estimate itself. For example, an estimated proportion of 80% might have a margin of error of 3%, meaning that we would conclude (with the appropriate confidence level, usually 19 times out of 20) that the true proportion is between 77% and 83%.

Suppose we want to estimate the proportion of Canadian livestock farms that store liquid manure. The estimated proportion is 14% with a standard error of 2.54. It can be deduced that the proportion of farms that do not store liquid manure is 86% and that the quality of the estimate is the same (i.e., the standard error is still 2.54). The standard error is an absolute error that applies to both the 14% and 86% estimates. The CV, being a relative error, would be different for the two estimates. It can even appear good for one proportion (86% for CV1) and bad for the complementary proportion (14% for CV2) as shown below:

CV1 = 100 * 2.54/86 = 3 (for farms which do not store liquid manure)
CV2 = 100 * 2.54/14 = 18 (for farms which store liquid manure)

Though the quality of the estimates is the same, the CV2 implies that the quality of the estimated proportion of farms which store liquid manure is much lower. In this case as with all proportion estimates, the CV can be misleading.

The following is a suggested CV rating system for level estimates, and a standard error (SE) rating system for proportion estimates:

CV Rating

0.01% - 4.99% A -- excellent
5.0% - 9.99% B -- very good
10.0% - 14.99% C -- good
15.0% - 24.99% D -- acceptable
25.0% - 34.99% E -- use with caution
35.0% and more F -- too unreliable to be published

SE Rating

0.01% - 2.49% A -- excellent
2.5% - 4.99% B -- very good
5.0% - 7.49% C -- good
7.5% - 12.49% D -- acceptable
12.5% - 17.49% E -- use with caution
17.5% and more F -- too unreliable to be published

Date modified: