Crop Condition Assessment Program (CCAP)

Detailed information for April to October 2022

The Crop Condition Assessment Program (CCAP) is a free web mapping application that provides weekly cropland and pasture condition reports across Canada and the northern United States, in near real-time based on analysis of low resolution satellite data. Historical conditions are also available at various levels of geography.

Data release - June 9, 2022 (First in a series of releases for this reference period.)

• Description
• Data sources and methodology
• Data accuracy

Description

The Agriculture Division of Statistics Canada has the mandate to collect census and survey information regarding all forms of agriculture in Canada, and provide it in an expeditious manner to clients, often government policy makers. Long ago Statistics Canada realized that new technologies such as satellite remote sensing and geographic information systems (GIS) could reduce costs and provide valuable information in support of its operations, without imposing additional response burden on producers.

The Crop Condition Assessment Program (CCAP), developed and maintained by the Remote Sensing and Geospatial Analysis Section (RSGA) within the Agriculture Division, is a prime example of such an application. The CCAP combines remote sensing, GIS, and the Internet to provide reliable, objective, and timely information on crop and pasture/rangeland conditions using a mapping application for the whole Canadian agricultural area and the northern portion of the United States.

The National Oceanic and Atmospheric Administration (NOAA) series of satellites carrying the Advanced Very High Resolution Radiometer (AVHRR) records images of the entire Earth's surface twice a day at one kilometre resolution. This detector captures two spectral bands (red and infrared) that have proven to be extremely useful for vegetation monitoring to produce the Normalized Difference Vegetation Index (NDVI).

Throughout the growing season from early April to mid October, on a weekly basis, Statistics Canada receives a 7-day composite of AVHRR images. Once the composites are received at Statistics Canada, some additional value-added processing is completed before the application is updated on the internet, normally on the same day that the data is received. This makes the CCAP an efficient tool to quickly and objectively depict agriculture conditions in near real time. Federal and provincial government agencies, grain marketing agencies, crop insurance companies, researchers and producers are typical users of the CCAP.

In 2010, the CCAP was further enhanced with the integration of the Moderate Resolution Imaging Spectroradiometer (MODIS) satellite data. At a spatial resolution of 250 metres and using its red and near infrared spectral bands, the MODIS sensor is able to show vegetation conditions (NDVI) with a higher accuracy than the AVHRR imagery.

RSGA has built an interactive mapping interface that allows users to view, via the Web, value-added satellite images and map products as well as charts and tabular data. Image products show vegetation conditions on a pixel by pixel basis while map products illustrate the predominant vegetation condition by regions as large as Census Agricultural Region, or as small as municipalities or townships. Current or historical conditions using the AVHRR-based application can be compared to the 1987-actual normal (2000-actual for the MODIS data) or to any other period of the available database.

Severe droughts, increasing competition among exporters, and the instability of crop products markets have underscored the importance of having accurate and timely information on crop conditions and potential yield. CCAP is able to supply the user community with frequent updates over a large geographic area well in advance of Statistics Canada's results of traditional surveys on crops.

Reference period: April to October

Collection period: Julian week 15 (which begins between April 6 and 12) to Julian week 41 (which begins between October 11 and 17)

Subjects

• Agriculture and food (formerly Agriculture)
• Crops and horticulture

Data sources and methodology

Target population

Crop and pasture land of all of Canada and the northern United States. The southern limit of the imagery and maps goes across the middle of the states of Washington and Idaho, southern Wyoming, the middle of Nebraska and Iowa, northern Illinois, southern Michigan, northern Pennsylvania, southern New York, and the middle of Connecticut and Rhode Island.

Instrument design

This methodology does not apply.

Sampling

This methodology does not apply.

Data sources

Data are collected from other Statistics Canada surveys and/or other sources.

Satellite imagery is received every Monday from early April until mid-October. Weekly updates are made to the web application within minutes of receiving the satellite data for near real-time use by the entire agriculture community.

The primary data source for the application is the Advanced Very High Resolution Radiometer (AVHRR), low resolution Earth observation satellite data, accessible to the general public. New in 2010, the application can be launched using a second data source. The original application is using 1-kilometre resolution data while the new application is using 250-metre resolution data.

- 1-kilometre resolution application: The data source is satellite imagery from the AVHRR sensor of the National Oceanographic and Atmospheric Administration (NOAA) satellite constellation. Data from the red (channel 1, wavelength of 580-680 nm) and near infrared (channel 2, wavelength of 725-1000 nm) channels are combined to produce a Normalized Difference Vegetation Index (NDVI). This data is collected daily during the growing season at the satellite receiving station in Prince Albert, Saskatchewan. The data are transferred to the Manitoba Remote Sensing Centre (MRSC), where they are input, pre-processed, geocoded, and resampled. A composite of processed images covering the latest seven-day period (Monday to Sunday) is produced to remove as much of the cloud effects as possible, which substantially improves the quantitative analysis capability for vegetation condition monitoring.

- 250-metre resolution application: The data source is satellite imagery from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor on-board the TERRA satellite. Data from the red (wavelength of 620-670 nm) and near infrared (wavelength of 841-876 nm) channels are combined to produce a Normalized Difference Vegetation Index (NDVI). The first step of data processing is completed at Agriculture and Agri-Food Canada, and consists of downloading every single image captured over Canada to produce a 7-day (Monday to Sunday) NDVI composite to remove cloud coverage as much as possible.

For both data sources, image composites are transferred to Statistics Canada for the rest of the value-added data processing, and for which results are used to update the web mapping application. The main steps to create the value-added products are:
- Download of the satellite composite data through FTP
- Reprojection of the imagery (MODIS images only)
- Image clipping and resampling (MODIS images only)
- Application of the cloud detection and removal algorithm
- Classification and creation of image products
- Average values calculations by geographic region
- Data quality assurance
- Web application files updates

Error detection

Errors relative to satellite data acquisition and image composites production are detected and corrected (if possible) by the organization that supplies the images. A visual inspection of the NDVI composites is completed at the time of data reception. After all data processing, another visual inspection is performed on the value-added data and compared with previous weeks of data and normal values. If errors are found, the data processing is repeated step by step to find the moment where the error occurred, so that the correction can be made at the right moment.

When receiving a composite image for a specified week, Statistics Canada performs the detection of cloud and other atmospheric residual contaminants for both AVHRR and MODIS images using an in-house programmed algorithm.

The vegetation index values normally follow a progressive curve, showing an increase at the beginning of the growing season, then levels off in the middle of the summer for a period of time that depends on the region, to finally show a decrease at the end of the growing season. Given the large relative size of image elements (pixels) of one square kilometre (NOAA) compared with the average field size in Canada, each pixel describes the average vegetation vigour of all fields covered by the pixel. For large fields, the 250 m * 250 m pixels of the MODIS images might allow vegetation conditions monitoring of a single field. The variation of NDVI values will most of the time be progressive, because even if a field is cut during one week, the remaining fields in the pixel will also influence the value of the resulting NDVI and will result in an average that will not rapidly fluctuate. Following the same idea, a field turns green in a matter of weeks, so again progressive change should be observed from week to week at the beginning of the season.

Based on this fact, a pixel will be considered contaminated if a sudden drop is immediately followed by a sudden increase in the following week.

In order not to have to wait for the following week to make a correction, a first detection will be made by comparing values of the current week with the previous week. The pixel will be corrected by repeating the value of the previous week if a NDVI value drops more than 0.05 from Julian week 15 to 28 (when values should normally increase or remain stable), or more than 0.20 from week 29 to the end of the season (when values usually drop). The threshold is higher after the normal peak of the NDVI values because at that time it is normal to observe larger drops in values because of the ripening and harvesting of annual and forage crops.

Upon reception of the following week's imagery, cloud detection and correction is performed again on the original values. The pixel will be identified as contaminated if a NDVI value drop of more than 0.01 is followed by an increase of at least 0.01.

For now, no correction is applied if atmospheric contamination is present for two or more consecutive weeks, because this requires a minimum of 4 weeks of data to perform, which would be difficult to operate in near-real time conditions.

Imputation

Imputation is employed when a pixel was identified as contaminated in the error detection step. The imputation method used is the following:

- When the following week data is not available, pixel values are temporarily replaced by the value of the previous week;
- When the following week data is available, the contaminated value is replaced by the average NDVI value of the previous and the following week.

Estimation

This methodology type does not apply to this statistical program.

Quality evaluation

There is no formal procedure for evaluating quality of the input and output data. However, images are visually inspected by remote sensing subject matter experts, upon reception of the satellite data to detect major errors in coverage or values.

After the cloud and other atmospheric contaminants removal algorithm process, the imagery is once again visually inspected for error detection.

The last step in error detection is completed after the creation of the value-added products. They are compared with products of the previous week to detect outliers in the progressive evolution of the vegetation indices in time. Products of the "Drought watch" application of Agriculture and Agri-food Canada (http://www.agr.gc.ca/pfra/drought/index_e.htm) are also used as a tool for quality assurance because the drought maps available on this site generally correlate well with the CCAP products.

In case of the detection of errors the event of error detection, the data processing will be repeated step-by-step from the reception of the raw data to find the step when the error occurred in order to apply necessary modifications at the right moment.

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects that could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

In order to prevent any data disclosure, confidentiality analysis is done using the Statistics Canada Generalized Disclosure Control System (G-Confid). G-Confid is used for primary suppression (direct disclosure) as well as for secondary suppression (residual disclosure). Direct disclosure occurs when the value in a tabulation cell is composed of or dominated by few enterprises while residual disclosure occurs when confidential information can be derived indirectly by piecing together information from different sources or data series.

Revisions and seasonal adjustment

This methodology type does not apply to this statistical program.

Data accuracy

The accuracy of the data released on the CCAP is dependant on the data quality of the satellite data supplied by the AVHRR and MODIS sensors, as well as the resulting composites images produced by the EODM software for the AVHRR composites, and the Agriculture and Agri-Food compositing software for the MODIS composites. A detailed description of the AVHRR sensor and data is available at https://www.usgs.gov/centers/eros/science/usgs-eros-archive-advanced-very-high-resolution-radiometer-avhrr-sensor?qt-science_center_objects=0#qt-science_center_objects. A description of the MODIS sensor and data is available at http://modis.gsfc.nasa.gov/about/.

The methodology developed to remove atmospheric contaminants from the raw NDVI composite data is not perfect, but it is possible to assert that it removes approximately 95% of the clouds and other contaminants in near real time.

Accuracy measures of the land cover data used as a filter for statistical calculations by region is available at https://www.agr.gc.ca/atlas/supportdocument_documentdesupport/annualCropInventory/en/ISO%2019131_AAFC_Annual_Crop_Inventory_Data_Product_Specifications.pdf. This land cover classification built with medium resolution (30 meters) satellite data produced an accuracy of around 85%. Combined with the fact that the agriculture, cropland and pasture filters are built from image pixels that are covered by at least 50% of the land cover class, it is likely that part of the land used for the NDVI average calculations is not covered by the desired class, which contaminates the values. The effect of this contamination is difficult to measure because it is influenced by the accuracy of the land cover classification, which can fluctuate from region to region, and from the nature of land cover classes that surround the agriculture areas (water, urban, forests, marshes, etc.), which also varies from region to region. However, the filters were built at much higher resolution than the NDVI pixels, which reduces the impact of this problem.

Aside from accuracy problems of the filters, the calculations made with the corrected satellite data are done using the entire agriculture, crop land or pasture area covered by all the regions (equivalent to the population), not on a sample basis.