Students With Inadequate Math Skills

Putting student math skills in context

The Programme for International Student Assessment (PISA) is an international assessment of the skills and knowledge of 15 year olds, coordinated by the Organisation for Economic Co-operation and Development (OECD). It assesses whether students approaching the end of compulsory education have acquired the skills that are essential for full participation in modern society, particularly in mathematics, reading, and science.¹ There have been four comparable PISA assessments of math skills—in 2003, 2006, 2009, and 2012.

PISA defines math literacy as:

an individual’s capacity to formulate, employ, and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena. It assists individuals to recognise the role that mathematics plays in the world and to make the well-founded judgments and decisions needed by constructive, engaged and reflective citizens.²

In its report on the 2012 PISA results, the OECD outlines the importance of math skills in today’s world:

An understanding of mathematics is central to a young person’s preparedness for life in modern society. A growing proportion of problems and situations encountered in daily life, including in professional contexts, require some level of understanding of mathematics, mathematical reasoning and mathematical tools, before they can be fully understood and addressed. Mathematics is a critical tool for young people as they confront issues and challenges in personal, occupational, societal, and scientific aspects of their lives.³

The importance of mathematical literacy was highlighted in a July 2013 press release from Canada’s Council of Ministers of Education, which stated that “numeracy was a key priority and that provinces and territories would work together to identify and share best practices on innovative teaching and learning strategies to raise student achievement in this area.”⁴

What are inadequate math skills?

PISA math skills are measured on a continuum, with level 6 the highest and level 1 the lowest. The Conference Board regards students as having inadequate math skills if they score at level 2 or below on the PISA math test. Although students performing at level 2 “can interpret and recognise situations in contexts that require no more than direct inference,...extract relevant information from a single source and make use of a single representational mode... [and] employ basic algorithms, formulae, procedures, or conventions,” their mathematical skills leave them poorly prepared to participate effectively and productively in Canada’s advanced economy.⁵

A student performing below level 1 is:

not able to show routinely the most basic type of knowledge and skills that PISA seeks to measure. Such students have serious difficulties in using mathematical literacy as a tool to advance their knowledge and skills in other areas. Placement at this level does not mean that these students have no mathematics skills. Most of these students are able to correctly complete some of the PISA items. Their pattern of responses to the assessment is such that they would be expected to solve less than half of the tasks from a test composed of only Level 1 items.⁶

How do the provinces rank relative to Canada’s international peers?

Taken as a whole, Canada compares reasonably well to international peers. With just under 35 per cent of Canadian 15 year olds scoring at level 2 or below on PISA’s math test, Canada achieves a grade of “B” and lags only Japan, Switzerland, the Netherlands, and Finland among international peers.

At a provincial level, one province—Quebec—earns an “A+” because it performs better on this indicator than the best-performing international country. British Columbia earns an “A,” while three provinces earn a “B” and three earn a “C.” Two provinces—Manitoba and Prince Edward Island—achieve only a “D.”

How do the provinces perform relative to each other?

In addition to ranking the provinces against Canada’s international peers, the provinces have been compared with each other and placed into three categories: “above average,” “average,” and “below average.”⁷

Quebec and British Columbia have the lowest proportions of students with inadequate math skills and are considered above average.

By contrast, more than half of students in P.E.I. have inadequate math skills, earning this province a below-average designation.

How have provincial inadequate math skills changed over time?

Between 2003 and 2012, the proportion of Canadian students with inadequate math skills increased in every province, in some cases dramatically. In 2003, Manitoba earned a grade of “B” relative to international peer countries, with only 30 per cent of students with inadequate math skills. But by 2012, 46.7 per cent of Manitoba students had inadequate math skills, pushing the province’s grade down to a “D.”

However, because many international peers also had increases in the proportions of their students with inadequate math skills, some provinces were able to maintain, or even improve, their relative grades. Although Quebec and British Columbia witnessed some increase in the proportion of students with inadequate math skills—0.3 and 6.3 percentage points respectively—the weaker performance of international peers allowed those provinces to improve their relative grades. Quebec climbed from an “A” to an “A+” grade, while British Columbia maintained an “A” grade. Similarly, while Saskatchewan’s proportion of students with inadequate math increased from 34.7 to 39.8 per cent, the province’s grade relative to international comparators remained a “B.”

Do PISA math performance results predict future educational success?

Results from the Youth in Transition Survey, by Statistics Canada, show a positive relationship between the math skills of 15-year-old students and the likelihood of completing further education. The effects of improving math skills are “in most cases statistically significant and also quantitatively important.”⁹

There were, however, gender differences. For girls, the effect of improving math skills had a stronger positive effect on high-school completion. For boys, the effect was strong on completing some post-secondary education.¹⁰

An outstanding issue is whether good results on the PISA math tests set students on a path to pursue advanced credentials in related fields. Over time, we might expect to see a relationship between these scores and the number of graduates in science, math, computer science, and engineering.

Footnotes

1    OECD, PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving, and Financial Literacy (Paris: OECD, 2013), 25.

2    Ibid., 23.

3    Ibid., 24.

4    Statistics Canada, Mathematics Proficiency Levels (accessed March 3, 2014).

5    Ibid.

6    No data were collected in the three territories and in First Nations schools.

7    To compare the performance of Canadian provinces relative to one another, we first determined the average score and standard deviation of the provincial values. The standard deviation is a measure of how much variability there is in a set of numbers. If the numbers are normally distributed (i.e., the distribution is not heavily weighted to one side or another and/or does not have significant outliers), about 68 per cent will fall within one standard deviation above or below the average. Any province scoring one standard deviation above the average is “above average.” Provinces scoring less than the average minus one standard deviation are “below average.” The remaining provinces are “average” performers.

8    OECD, Pathways to Success: How Knowledge and Skills at Age 15 Shape Future Lives in Canada (Paris: OECD, 2010), 67.

9    Ibid.