International Adult Literacy and Skills Survey (IALSS)

Detailed information for 2003

Status:

Active

Frequency:

Occasional

Record number:

4406

The International Adult Literacy and Skills Survey (IALSS) is the Canadian component of the Adult Literacy and Life Skills Survey (ALL). The main purpose of the survey was to find out how well adults used printed information to function in society. Survey data include background information (demographic, education, language, labour force, training, literacy uses, information and communication technology, income) and psychometric results of respondents' proficiency along four skill domains: prose and document literacy, numeracy and problem-solving.

Data release - May 11, 2005 (International Report); November 9, 2005 (National data); November 30, 2005 (National Report)

Description

The International Adult Literacy and Skills Survey was a seven-country initiative conducted in 2003. In every country nationally representative samples of adults aged 16-65 were interviewed and tested at home, using the same psychometric test to measure prose and document literacy as well as numeracy and problem-solving skills.

In Canada, the survey population was expanded to provide information on respondents over the age of 65. The main purpose of the survey was to find out how well adults used printed information to function in society. Another aim was to collect data on the incidence and volume of participation in adult education and training, and to investigate the relationships between initial and adult education, on the one hand, and literacy, numeracy and problem-solving proficiency and wider economic and social outcomes, on the other.

In addition, a subsidiary goal was to provide information regarding change in the distribution of skills over the years since the previous survey (the 1994, International Adult Literacy Survey - to access the 1994 IALS metadata, please use the "Other reference periods" link in the sidebar above). The link between the two measures was made by using items from the 1994 study in the design of the 2003 study.

Users of the data include federal and provincial governments, academics, literacy and skills development professionals, media and interested members of the public. The data are used to inform policy decisions, help effectively allocate resources where needed and inform decisions on the composition and content of remedial skill development course and adult education.

Subjects

  • Adult education and training
  • Education, training and learning
  • Literacy

Data sources and methodology

Target population

This is a sample survey with a cross-sectional design representing Canadian adults aged 16 and over not residing in institutions or on Aboriginal reserves. In addition to provincial and territorial estimates, the survey was designed to provide reliable estimates for a variety of special target populations such as recent and established immigrants, Francophones in New Brunswick, Manitoba and Ontario, Anglophones in Quebec, Urban Aboriginals in Manitoba and Saskatchewan, Youth in Quebec and British Columbia and Aboriginal residents in the three northern territories.

The population that was in fact covered by the survey differed in a number of practical respects, but all of these exclusions combined are still within the survey standard of no more than 2% of the total population. Residents of sparsely populated regions were excluded from the survey population. It is estimated that the coverage for the survey was 98.5% nationally, with provincial coverages ranging from 95% to nearly 100%. In the northern territories, reduced levels of coverage (70-90%) were obtained.

Random stratifiers from the 2001 Population Census identified households to be contacted for the survey. The Census also allowed the efficient identification of households with a probability of containing respondents of each of the special target populations. Once a chosen household was contacted, a respondent over the age of 16 was randomly interviewed and completed a paper and pencil test. Further selection criteria such as mother tongue, immigration status and Aboriginal status were instituted for households chosen in one of the special target population samples.

Instrument design

The survey questionnaire and psychometric task booklets were designed by groups of experts from around the globe. Each task items was translated into English and this version provided the master items that was to be adapted by each of the participating countries into their own language. All of the instruments were tested in a pilot survey conducted in the fall of 2001 and the final task booklets were created using the items providing the most reliable and stable parameters in all four domains. Many of the background questions and a selection of prose and document domain tasks asked in this 2003 survey trace their origins to the 1994 International Adult Literacy Survey. This was done in order to provide some psychometric link that would allow some limited comparisons of skill distribution over time.

The survey instruments included a background questionnaire administered in the respondent's home by a Statistics Canada interviewer, a short screener task booklet that was made up of six simple literacy tasks. If the respondent could successfully complete three out of the six questions they would then be given a longer task booklet composed of stimuli and questions related to the four skill domains (prose and document literacy, numeracy and problem-solving). The items for each domain were placed in blocks of approximately 15 items (two blocks of prose, two blocks document, two numeracy and two problem-solving). These blocks were rotated so that 28 booklets were constructed each containing three blocks. These blocks were combined into booklets in such a way that each block was paired with the others in the first, second and third position. Each respondent was randomly assigned one of the 28 booklets. Using a 3-PL IRT model, the respondents on the final data file have been assigned plausible proficiency values ranging from 0 to 500 in each of the measured domains.

The Background Questionnaire collected information on ethnicity, immigrant status, education (extensive section), linguistic information, self-assessment of reading and writing in mother tongue, parental education and occupation, work status and history, occupation, industry, workplace literacy, numeracy and problem-solving practices, wages, adult education and learning (extensive section), literacy and numeracy practices at home, civic engagement, mental and physical health, information and communication technology use and familiarity, household information (e.g. size of family, number of books at home, reading practices with children) and income.

Sampling

This is a sample survey with a cross-sectional design.

The IALSS used a cross-sectional multi-stage sample design. The sampling unit was the household. The sampling frame was the 2001 Census of Population (reference date, May 15th). A stratified multi-stage probability sample design was used to select the sample from the census frame.

Each province was divided into two strata: an urban stratum consisting of larger urban centres, and a rural stratum consisting of the remaining smaller urban centres and rural areas.

Within the urban strata, two stages of sampling were used. In the first stage of sampling, dwellings were selected systematically with probability proportional to household size. In the second stage, one individual was randomly selected from the list of eligible household members residing in the selected dwelling at the time of data collection.

In order to reduce collection costs, three stages of sampling were used in the rural strata. The rural stratum of each province was partitioned into smaller geographic areas called clusters. The first stage of sampling consisted of selecting clusters with probability proportional to population size. The second stage of sampling consisted of selecting dwellings from within each selected cluster. Like the urban strata, the final stage of sampling consisted of the random selection of one eligible individual from each selected dwelling.

A base sample of 16,000 dwellings was selected to cover the general population. An additional 24,000 dwellings were selected in supplementary samples targeting the following subpopulations: Francophones in New Brunswick, Ontario and Manitoba; Anglophones in Québec; immigrants in Québec, Ontario, Alberta and British Columbia; youth (16-24) in Québec, youth (16-29) in British Columbia; Aboriginal persons in urban Manitoba and Saskatchewan; Aboriginal persons and non-Aboriginal persons in Yukon, the Northwest Territories and Nunavut.

Data sources

Data collection for this reference period: 2003-04-03 to 2003-08-20

Responding to this survey is voluntary.

Data are collected directly from survey respondents.

Data were collected in the home of each respondent. Whenever possible, contact and screening was done by phone to confirm the household address and make an appointment with the chosen respondent to visit them at their convenience. Proxy responses were not permitted. The Background Questionnaire was administered by the interviewer using computer-assisted interviewing techniques. Once completed a short booklet of six simple tasks (Core Task booklet) was provided along with a pencil and eraser. The answers provided in the booklet were scored by the interviewer and, if the respondent correctly answered three out of the six questions, they were offered a longer task booklet. In all, 28 task booklets were constructed by combining two blocks of items from a pool of eight blocks (four measuring prose and document literacy, two measuring numeracy and two measuring the problem-solving domain). Each block had on average about 40 questions related to about 15 specific stimuli (additional testing material such as a newspaper, calculator, ruler and templates were provided whenever appropriate). The combination of blocks into 28 booklets was done so that each block appeared in a booklet paired with each of the other blocks in both first and second position. Each respondent was given as much time as they required to answer as many questions as they could in one of the booklets. The booklet to be answered by any given respondent was randomly pre-assigned for each sampled household. The completed booklets were then sent to Statistics Canada and scored by experts trained to follow the international scoring criteria. Examples of the tasks found in these booklets are included in Annex B of the free publication "Building on our Competencies: Canadian Results of the International Adult Literacy and Skills Survey" (available through the online catalogue number 89-617-XIE or accessible through the link "Publications" in the side bar menu above).

View the Questionnaire(s) and reporting guide(s) .

Error detection

The IALSS was collected using a computer-assisted survey application. As such, many of the error detection and editing took place during collection. Validation of values outside specified ranges was performed by the interviewer whenever they were flagged by the Blaise application on the computer and the application automatically directed the flow of the questionnaire based on pre-arranged logic and the respondent's previous answers.

Once the data were collected and transmitted to head office, three phases of error detection were initiated. The first was a general clean-up of the data to accomplish the following goals: 1) remove duplicate records from the file, 2) verify the Background Questionnaire against the sample file, 3) verify the integrity of the status code, 4) identify missing records, and, 5) create a response file.

The editing phase of the data processing included a series of edit steps to be performed. First, a top-down flow edit cleaned up any paths that may have been mistakenly followed during the interview. This step was followed by consistency edits for certain key variables. This step assured concordance between variables such as age, year of immigration, number of years of formal education, age when the respondent took a type of training, and age when the respondent completed his/her highest level of education. The third quality assurance step involved imputation of certain values and the derivation of variables to ease analysis and respect the confidentiality of our respondents.

Imputation

Missing data were imputed for a small number of variables using the Impudon software. Nearest neighbour imputation was used, which involves copying information from a "donor" record with similar characteristics.

The task language was imputed for respondents who did not complete a task booklet; the imputation was performed for approximately 9% of records. Exact personal income and household income were imputed for records which reported a range for income rather than a value. As well, age, level of education, year of immigration and Aboriginal status were imputed for a very small number of records, fewer than 0.1% of records overall.

Estimation

Estimates are produced using weights attached to each sampled unit. The weight of a sampled unit indicates the number of units in the population that the unit represents. The weights were calculated in several steps:

1) An initial weight was calculated based on the probability of selecting the unit in the sample.

2) The weights were adjusted to account for non-response.

3) The weights from the base sample and the various supplementary samples were integrated using a multiple-frame method.

4) Generalized regression (GREG) estimation was used to calibrate the weights and to make them agree with external population counts.

The quality of the estimates is assessed using estimates of their coefficient of variation (CV). Given the complexity of the IALSS survey design, CVs cannot be calculated using a simple formula. Jackknife replicate weights are used to establish the CVs of the estimates.

Quality evaluation

To ensure high quality data, International Survey Administration Guidelines were followed and supplemented by adherence to Statistics Canada's own internal policies and procedures.

The interviews were conducted in homes in a neutral, non-pressured manner. Interviewer training and supervision were provided, emphasizing the importance of precautions against non-response bias. Interviewers were specifically instructed to return several times to non-respondent households to obtain as many responses as possible. Their work was supervised using frequent quality checks, especially at the outset of data collection.

As a condition of participation in the international study, it was required to capture and process files using procedures that ensured logical consistency and acceptable levels of data capture error.

Persons charged with scoring received intense training in scoring responses to the open-ended items. To ensure quality, monitoring was done in two ways. First, at least 20% of the tasks were re-scored. The two sets of scores needed to match with at least 95% accuracy before the next step of processing could begin. Second, an international re-score was performed. Each country had 10% of its sample re-scored by another country. Strict accuracy was demanded: a 90% correspondence was required before the scores were deemed acceptable.

A study was undertaken to uncover the cause of discrepancies between the distribution of the Level of Education variable observed in IALSS and that from the Census. The study concluded that the comparability of the two survey estimates had been impacted considerably by the combined effects of the differences in questionnaire wording and collection modes used (personal interviews vs. self-completed enumeration).

Disclosure control

Statistics Canada is prohibited by law from releasing any information it collects which could identify any person, business, or organization, unless consent has been given by the respondent or as permitted by the Statistics Act. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Revisions and seasonal adjustment

This methodology type does not apply to this statistical program.

Data accuracy

Coverage:
It is estimated that the coverage for the survey was 98.5% nationally, with provincial coverages ranging from 95% to nearly 100%. In the northern territories, reduced levels of coverage (70-90%) were obtained because only the communities covered in the national Labour Force Survey were included (personal interviewing outside of these centres was considered either too costly or impractical).

Response Rate:
The response rate was 65.6% nationally, with provincial rates ranging from 59.5% to 75.4%.

Coefficient of Variation (CV):
The quality of the estimates is assessed using estimates of their CVs. Jackknife replicate weights are used to establish the CVs of the estimates. The following guidelines are recommended:
- If the CV is less than 16%, the estimate can be used without restriction;
- If the CV is between 16% and 33%, the estimate should be used with caution;
- If the CV is 33% or more, or if the estimate is based on fewer than 30 observations, then the estimate should not be released.


The table below shows the CVs for the percent of the population at each prose proficiency level:

Date modified: