Tag Archives: research

Why is PISA getting such a bad rap lately?

I have been reading a great deal of educational comment recently questioning what is described as an obsession with PISA scores. PISA is the OECD’s three year survey of educational achievement of  15 year olds in science, mathematics and reading. Let’s get one thing clear from the start.

The criticisms have more to do with us and how we are using PISA, than it has to with the PISA assessments and data themselves.

Most of the criticism of PISA seems to be coming from the USA where there is an epidemic of standardized testing that has swept across the country, causing a significant malaise in education. In the USA we are looking at PISA as if it is yet another high stakes test. Viewed this way the stakes are indeed very high, because it is the nation’s education system that is been assessed – and found wanting. In Australia there is similar criticism, perhaps because although the nation’s 15 year olds still perform well above the OECD average, their edge is slipping.

I guess it is understandable that when you see a test is not treating you well, the first thing you want to do is find fault in the test. The problem is that we are focusing on the wrong things.

Finland has been lauded internationally for some time now because of its student achievement levels on all three PISA measurements – science, mathematics and reading. I listened recently to Dr Pasi Sahlberg, a life-long educator who was traveling in Australia earlier this year. He is currently the General Director of the Centre for International Mobility and Cooperation, Finland and previously he was a Senior Education Specialist with the World Bank as well as the Director of the Centre for School Development, Helsinki.

Finland, it seems has a very different attitude to PISA and, in fact, to all forms of standardized testing. PISA testing just happens to be something that they do. It is not viewed with anything like the significance that countries caught up in the Global Educational Reform Movement (GERM) give it. In Finland students do not take any externalized standardized tests until they have finished high school – at around 18 years of age. They do not judge the quality of their education using PISA data, but they do acknowledge that PISA data reinforces what they already knew about their education – that they had excellent teachers and successful schools.

And they knew all this without ever having resorted to widespread standardized testing.

The problem with our way of looking at PISA is that we focus on the numbers (are we above or below the OECD average?) and the ranking (who is beating us?). Our efforts then are directed towards trying to improving the score, competing with the others and moving up the ranks (the Prime Minister of Australia is determined to have Australia back in the top 5% of PISA by 2025). We start asking questions about how to tighten and standardize the curriculum, how to ensure that everyone is teaching the things that are going to be tested, how to give our students the test taking skills they will need to be successful in the tests.

But PISA is far more than ranking tables and scores. If only we would take the time to examine the data we would learn much more about how to improve our education systems. We would discover that we will achieve nothing if we continue to focus on accountability and standardizing curriculum.

We know from PISA  that in the 2006 round of testing less than 10% of the variation in student performance was explained by student background in five of the seven countries with the highest mean science scores of above 530 points. PISA demonstrates that equity and performance are highly related. In Finland, equity is considered more important than excellence. Dr Sahlberg tells us that “we have systematically focused on equity and equality in our education system, and not so much on excellence and achievement like many other countries have done.”

What are we doing to ensure that all our young people have access to the same quality of education regardless of socio-economic circumstances? In Finland it is illegal to charge fees for any education program that leads to a qualification, because education is deemed to be a right for all its citizens. There are no private schools in Finland. If we only looked more closely at the PISA data we would see that equity, not accountability or curriculum, is foundational to high achievement.

In Finland it is virtually impossible for a ‘bad’ teacher to enter the profession. The demand for teacher training positions is so high that last year 2,500 people competed for 120 positions and the selection panel was able to cherry pick the best of the best. Teachers are not paid dramatically high salaries, but they are highly respected. According to Dr Sahlberg “it’s more difficult to get into primary school teacher education in Finnish universities than medicine.”

We could also learn much about student attitudes towards specific curriculum areas as well as their views on the world. For example, a majority of students reported in PISA 2006 that they were motivated to learn science, but only a minority reported interest in a career involving science. This information needs to be fleshed out if we are hoping to create a nation of innovation and creative development in the immediate future. Why is it that one significant feature of a student’s background in terms of science achievement was whether they had a parent in a science-related career? What is the significance of the fact that PISA reveals USA 15 year olds did not do well in mathematics and yet they feel very confident in their mathematical abilities?

It is disturbing to discover that there is some degree of pessimism among the students about the future of the natural environment. On average across OECD countries, only 21% of students reported that they believed the problems associated with energy shortages would improve over the next 20 years. Are we taking this into account when we review our educational priorities?

PISA is a powerful resource if we would dig more deeply and use it to do something more than hijack the ranking tables to justify a test taking industry that is capitalizing on our failure to think below the surface. In New York, Pearson Education currently has a five-year, $32 million contract to administer state tests and it creates and sells education programs seamlessly aligned with the high stakes standardized tests for students and teacher assessments they are also selling. Pearson reported revenues of approximately $9 billion in 2010. This is big business.

What a tragedy if the most tangible outcome of a comprehensive review such as the OECD’s PISA was to be the creation of a multi-billion dollar industry rather than a successfully educated generation of our nations’ children.


Filed under Language and literacy, Testing, Thinking

Distractions and Engagement

Everywhere I go now there are TV screens. In the doctor’s waiting room, at the car repairers, in restaurants, at airports. Everywhere. It’s as though we are being told that we don’t know how to amuse ourselves with our own thoughts or with a book.

I remember the great conversations we had as a family when our kids were small. We would go out to dinner every now and then and the new environment, the people around us, the occasion itself were spurs to some great discussions, lots of laughter and questions, questions, questions from the kids. They were curious and intrigued by everything. We relished the opportunities to interact with them and get them thinking about what they saw and heard.

I was in a restaurant last week and at the adjacent table was a family of five. The parents had their eyes pretty much glued to the TV screen on the wall in front of them, two of the kids were profoundly involved with their phones and the third just sat gazing into the middle distance. What else could he do? There was no one to talk to.

The disturbing thing was that my phone was sitting on the table on front of me, and it buzzed. My hand went almost instinctively to pick it up, to check out what piece of utter trivia or earth shattering importance was waiting for me. We were in the lull between having placed our order and receiving it. That space where you dip your bread in the olive oil and balsamic and try not to take too much edge off your appetite. It was the time when you check out the other diners, rearrange your napkin, ponder whether dessert is a likely option tonight. I knew if I picked up my phone, my companion would probably pick up his and there we would be, both tied by our eyeballs to our phones.  Conversation potential zilch!

When did we forget to talk to each other? When did the world around us decide that we could no longer be occupied with our own thoughts and the thoughts of others? Why are we allowing this to happen?

I think we are in real danger of forgetting how to engage – to engage with ourselves and with others. I fear that we are becoming immersed in a sea of distractions that tug us away from one another, and away from what goes on in our own heads. On our phones we flick from email, to twitter, to facebook, and back again. On our computers we hit link after link, scanning articles, leaving them half read and hardly considered as we move on to the next fascinating throw away tag line.

I think we need to learn a whole new set of manners about smart phone use in public. I also believe we need to teach our kids and ourselves to resist the temptation to flit about like intellectual butterflies from one distraction to another, from one hyperlink to the next. We need to teach our kids and ourselves how to engage.

It is possible to set an Ipad so that the opportunities for browsing are limited. With ‘Guided Access’ the teacher or parent can restrict usage to a single application and control which features are available. If your kids are using Ipads at home or at school for a specific purpose you can encourage them to stay on task, in focus and not be tempted to flutter off to the next pretty flower.

My son tells me of a great smartphone game when groups go out to dinner together. Everyone puts their phone in a pile in the center of the table, one on top of the other. The first person to pick up a phone should it ring or buzz has to buy a round of drinks.

Students easily become overloaded by the sheer volume of sources available to them when researching on the internet. Instead of allowing them to jump from source to source, teach them how to thoroughly investigate one or two. Show them how to determine the source of the information, encourage them to read to the end, ask them to compare and evaluate two sources on the same subject, for example www.crazydogtheoriesonhealth.com and www.mayoclinic.com or www.theearthreallyisflat.org and www.nationalgeographic.com.

Start a movement! The Bring Back Conversations Movement. Create a T shirt. Ask for a table where you can sit with your back to the TV in a restaurant. Design a bumper sticker. Start a conversation group.  Do whatever it takes to encourage people to communicate with clarity and precision and to listen with empathy and understanding.

In the book Nurture Shock by Po Bronson and Ashley Merryman I discovered that simply placing children in desegregated schools doesn’t appear to lead to greater racial interaction. Young children when dressed in either a red or a blue t-shirt at school will eventually choose their friends from those kids wearing the same color shirt. It seems to be that only when we talk about something can we begin to influence behavior. We need to talk to our kids about the need for conversation. We need to teach them how to participate. We should be encouraging them to explore the contents of their own mind and the minds of others and then to share their ideas.

It isn’t one thing or another. We should be teaching our kids how to be sociable users of social media. We should also be teaching them how to interact and communicate without always needing an electronic gadget as an intermediary.


Filed under Thinking

Where Are The Poets and Where is Einstein?

In a recent blog I was fretting about the future of poetry. The problem is    actually wider and deeper.

Einstein said:

 “When I examine myself and my methods of thought, I come close to the conclusion that the gift of imagination has meant more to me than any talent for absorbing absolute knowledge.”

He also said of his most famous discovery:

“The theory of relativity occurred to me by intuition, and music is the driving force behind this intuition”.


And so the inevitable questions are:

  • How do we test for imagination using a multiple choice Scantron sheet?
  • What are we doing to teach more than just the acquisition of factual knowledge?
  • Why is the teaching of music in decline?

A USA report by the Center on Educational Policy in December 2007 found that 62% of schools had increased the instructional time spent on the two major tested areas of English Language Arts and mathematics. It also found, unsurprisingly, that 31% of schools had reduced the time given to non-tested subjects. Another five years has made things worse. Some school systems, faced with budget cuts, have eliminated specialist music teachers from elementary schools altogether.

The reduction in time given to non-tested areas was found to be more prevalent in school districts that were considered in need of improvement. “In need of improvement” can virtually always be read as “high poverty”. If the trend continues we can expect to see the Arts become curriculum offerings for the affluent while the poor will be more and more denied the opportunity to share in these riches of our society, in the same way they are denied the more material riches.

The high stakes testing regime has been alive and kicking in the USA since 2001 and has had plenty of time to do its work on the Arts. In Australia the Federal government, for reasons known only to itself, and not to educators, is heading down the testing path of the USA – as usual about ten years later and just as the USA begins to wake up in more enlightened circles to the fact that this path leads nowhere desirable.

I was disturbed to find that although there was a weekly specialist music lesson, there was virtually NO classroom music in the school I lead in the USA. There was a smattering in the very youngest classes, but nothing more. I recall the astonishment on the faces of the kids and the teacher of one of my grade three classes when I went into the room and had them all stand up, follow me, and learn the song and actions to “A Pirate Went To Sea, Sea, Sea”.

When I was a classroom teacher my classrooms were always filled with music. The CD player was on my desk, we had quiet classical music in the background during writing sessions, we sang songs about numbers, about countries, about ideas. Music was a part of how the children learned.

Often the music was also associated with movement, so we were on our feet singing counting songs, clapping, jumping and stamping our feet as we sang our multiplication tables. I would make up short refrains to help children to learn how to spell difficult words so we could sing them as well as write them and recite them.

I interacted with the music teacher so that she could adjust her music program to expand on the things we were learning about in other curriculum areas, and so that we could sing the songs and play the music she had been working with when the children returned to my classroom.

Here is the problem. As the Common Core State Standards and the Australian Curriculum are implemented in their respective school systems, we can anticipate a flood of testing to follow close behind. Results will be made public in the name of some sort of ‘accountability’ – often more a search for someone to blame. That will increase the stakes for these tests and the inevitable slide will accelerate, as tested subjects take up more and more curriculum time and the non-tested arts subjects are relegated to the frills area of the curriculum.

We need to take at least two kinds of action.

Firstly we need to protest loudly and often if and when we see any diminution in the time and value given to the arts in our schools.

Secondly, we need to find ways to incorporate the kinds of learning that are facilitated by the use of images, non-linguistic representations, the translation of knowledge into visual media and the use of music in our classrooms, whatever we are teaching.

If specialist arts lessons are reduced, let’s bring them into all the areas of the curriculum that will be tested. Let’s teach our kids mathematics, poetry, grammar, science, social studies – the whole panoply – with color, with images, with music and with movement.

We need poets and we need more Einsteins.




Filed under Thinking

A Vaccine for the Internet Virus

Do you find emails in your in box that make you gasp with amazement that something so terrible could actually be true? Are you amazed to discover that one of your political leaders is even more unscrupulous than you had suspected? Are you disturbed to discover that your worst fears are supported and that there is, in fact, a giant conspiracy operating? Or are you just sleeping less soundly at night because the Mayan’s have predicted the end of the world in 2012?

These emails are often the result of well-meaning friends reposting something they came across on the internet. They repost it to their friends, their friends repost it to their friends and on and on – suddenly it’s gone viral because we were all vectors for infection!

Maybe there has never been a more important time for us to deepen our thinking. One of the Habits of Mind tells us to gather data through all our senses. I would like to add a little to that and suggest we need to gather data through all our senses AND all our sources. And keep in mind that it is data we need to be gathering, not just opinion.

These emails are often filled with a kind of data – quotes, descriptions of events, sentences from press releases – but I remember being taught at school to go back to primary sources and documents whenever possible. And why is this? Because it is so very easy to completely distort the message if you change the context. Let’s take a recent case. The author and television presenter  Clive James, was recently interviewed by Radio 4 in London and in the course of the interview he mused on his own mortality and the fact that he was a lot closer to the end than to the beginning. This interview with its nuances, tones of voice, pauses, laughs and the like was then taken up by a print journalist from the Daily Mirror and without all these subtleties his written article looked as though James was teetering on the edge of the grave. http://www.telegraph.co.uk/culture/tvandradio/9349291/Clive-James-Im-not-dead-yet.html. James was highly irritated by the rumours of his imminent death that this article sparked.

Teachers encourage their students to question what they read and hear. They are skilled in helping young people drill through the surface layers and get to the sources. One of my most useful web sites to help with this task is www.snopes.com. Here you can type in the latest rumour or viral ‘fact’ and begin to test its veracity and reliability. Snopes will provide information about where it came from and what other information is available to help us decide if it is ‘true’, ‘false’ or ‘undecided’.

Skilled thinkers ask questions and take in data through all their senses and from all their sources. The internet serves the rumour mill like nothing ever has before. Thinking is the tool that can apply a vaccine to control viral emails. Think before you hit the ‘send’ button.

1 Comment

Filed under Thinking