Tag Archives: international comparisons

Doing it by the numbers

I am very puzzled by the debate surrounding the release of the PIRLS Report and Australia’s poor performance in grade 4 reading. Two years ago the PISA Report showed Australia firmly in the top 10% of OECD nations, although we had declined since the previous round of testing. the USA performance was lack luster and well below that of Australia and New Zealand. Now, two years later, the positions have been reversed. The USA is respectable and Australia is appalling – according to this new set of numbers.

So what happened in two years? Did Aussie kids suddenly forget everything they knew two years earlier? I know that the PISA tests look at 15 year olds, but they were fourth graders before they became fifteen year olds.

Finland and the usual suspects among the Asian Tigers are still up the top. But what’s going on in the USA and Australia/NZ?

Can someone help me here?

Leave a comment

Filed under Teacher education, Testing, Thinking

Why is PISA getting such a bad rap lately?

I have been reading a great deal of educational comment recently questioning what is described as an obsession with PISA scores. PISA is the OECD’s three year survey of educational achievement of  15 year olds in science, mathematics and reading. Let’s get one thing clear from the start.

The criticisms have more to do with us and how we are using PISA, than it has to with the PISA assessments and data themselves.

Most of the criticism of PISA seems to be coming from the USA where there is an epidemic of standardized testing that has swept across the country, causing a significant malaise in education. In the USA we are looking at PISA as if it is yet another high stakes test. Viewed this way the stakes are indeed very high, because it is the nation’s education system that is been assessed – and found wanting. In Australia there is similar criticism, perhaps because although the nation’s 15 year olds still perform well above the OECD average, their edge is slipping.

I guess it is understandable that when you see a test is not treating you well, the first thing you want to do is find fault in the test. The problem is that we are focusing on the wrong things.

Finland has been lauded internationally for some time now because of its student achievement levels on all three PISA measurements – science, mathematics and reading. I listened recently to Dr Pasi Sahlberg, a life-long educator who was traveling in Australia earlier this year. He is currently the General Director of the Centre for International Mobility and Cooperation, Finland and previously he was a Senior Education Specialist with the World Bank as well as the Director of the Centre for School Development, Helsinki.

Finland, it seems has a very different attitude to PISA and, in fact, to all forms of standardized testing. PISA testing just happens to be something that they do. It is not viewed with anything like the significance that countries caught up in the Global Educational Reform Movement (GERM) give it. In Finland students do not take any externalized standardized tests until they have finished high school – at around 18 years of age. They do not judge the quality of their education using PISA data, but they do acknowledge that PISA data reinforces what they already knew about their education – that they had excellent teachers and successful schools.

And they knew all this without ever having resorted to widespread standardized testing.

The problem with our way of looking at PISA is that we focus on the numbers (are we above or below the OECD average?) and the ranking (who is beating us?). Our efforts then are directed towards trying to improving the score, competing with the others and moving up the ranks (the Prime Minister of Australia is determined to have Australia back in the top 5% of PISA by 2025). We start asking questions about how to tighten and standardize the curriculum, how to ensure that everyone is teaching the things that are going to be tested, how to give our students the test taking skills they will need to be successful in the tests.

But PISA is far more than ranking tables and scores. If only we would take the time to examine the data we would learn much more about how to improve our education systems. We would discover that we will achieve nothing if we continue to focus on accountability and standardizing curriculum.

We know from PISA  that in the 2006 round of testing less than 10% of the variation in student performance was explained by student background in five of the seven countries with the highest mean science scores of above 530 points. PISA demonstrates that equity and performance are highly related. In Finland, equity is considered more important than excellence. Dr Sahlberg tells us that “we have systematically focused on equity and equality in our education system, and not so much on excellence and achievement like many other countries have done.”

What are we doing to ensure that all our young people have access to the same quality of education regardless of socio-economic circumstances? In Finland it is illegal to charge fees for any education program that leads to a qualification, because education is deemed to be a right for all its citizens. There are no private schools in Finland. If we only looked more closely at the PISA data we would see that equity, not accountability or curriculum, is foundational to high achievement.

In Finland it is virtually impossible for a ‘bad’ teacher to enter the profession. The demand for teacher training positions is so high that last year 2,500 people competed for 120 positions and the selection panel was able to cherry pick the best of the best. Teachers are not paid dramatically high salaries, but they are highly respected. According to Dr Sahlberg “it’s more difficult to get into primary school teacher education in Finnish universities than medicine.”

We could also learn much about student attitudes towards specific curriculum areas as well as their views on the world. For example, a majority of students reported in PISA 2006 that they were motivated to learn science, but only a minority reported interest in a career involving science. This information needs to be fleshed out if we are hoping to create a nation of innovation and creative development in the immediate future. Why is it that one significant feature of a student’s background in terms of science achievement was whether they had a parent in a science-related career? What is the significance of the fact that PISA reveals USA 15 year olds did not do well in mathematics and yet they feel very confident in their mathematical abilities?

It is disturbing to discover that there is some degree of pessimism among the students about the future of the natural environment. On average across OECD countries, only 21% of students reported that they believed the problems associated with energy shortages would improve over the next 20 years. Are we taking this into account when we review our educational priorities?

PISA is a powerful resource if we would dig more deeply and use it to do something more than hijack the ranking tables to justify a test taking industry that is capitalizing on our failure to think below the surface. In New York, Pearson Education currently has a five-year, $32 million contract to administer state tests and it creates and sells education programs seamlessly aligned with the high stakes standardized tests for students and teacher assessments they are also selling. Pearson reported revenues of approximately $9 billion in 2010. This is big business.

What a tragedy if the most tangible outcome of a comprehensive review such as the OECD’s PISA was to be the creation of a multi-billion dollar industry rather than a successfully educated generation of our nations’ children.

12 Comments

Filed under Language and literacy, Testing, Thinking