Pook's Perplex

Friday 27 January 2012

Fifteen years or so ago, George Pook, the then Head of Assessment at the IB, carried out an interesting experiment. He took 1000 English A1 exam scripts, and paid a couple of students to go through them and count laboriously all the punctuation – full stops, colons, inverted commas, the lot. He then compared the total of punctuation in each script with the actual grade awarded by the normal marking procedure. Guess what? He found that there was a very close correlation – the more punctuation you used, the more likely you were to have received a high grade.

 

He took these results to a planning meeting with Very Important IB People (VIIBPs), in which the IB's examining strategies were to be reviewed. Pook being Pook – a quiet man with an exceptionally incisive mind and a sharp sense of humour – he impishly suggested (I'm told) that this proved that IB could actually dispense with highly qualified English examiners, and just have unqualified labour counting punctuation. I believe that this wasn't his true purpose – he simply wanted to question the way that exams are traditionally carried out. We set certain types of standardised tasks, to be completed in standardised time periods, and assume that because this system has been acceptable in the past, it is acceptable now. But do such tasks accurately evaluate students' abilities? And even if they do, are such tasks necessarily the best way to evaluate ?

 

Actually, Pook's Punctuation Paradigm (I can't resist the alliteration) indicates a convincing insight. The whole point of punctuation is to demonstrate the structure of ideas: to separate ideas, link them, and show their relative value and importance. If so, you would expect a student who has a lot of highly differentiated ideas, elaborately connected, to use a lot of punctuation to handle such complexity. Equally, a student who has few ideas, poorly thought out, will not need much punctuation beyond the occasional comma or full stop (and we all know students like that!). Punctuation; is not – good “in” itself of – course / unless. It is : linked 'to' coherent (thought) – but punctuation can surely be seen as an indicator of developed thinking.

 

Now, I'm sure that Pook was not proposing to replace Paper 2 Written Production with Paper 2 Punctuation! Apart from anything else, the minute the students thought that marks were directly linked to punctuation, they would play the game and punctuate obsessively. However, this whole anecdote points us towards the fundamental purpose of Paper 2 – to test students' ability to use developed language to express developed thinking. The essence of exam tasks in English B Paper 2 should be to generate indicators of those two inter-related skills, which are the true transferable skills that are valuable for “how individual students perform in real life”. All the rest of the exam mechanism – contexts, subject & topics, audiences – is set-dressing, really.

 

This of course illustrates my whole approach to English B, as indicated in this website: that the teaching of language and the teaching of good thinking have to go hand in hand. And wouldn't it be charming to think up a whole radically different way of testing language competence which didn't depend on arbitrary tasks on an arbitrary day … ?


Tags: assessment, language, teaching

On hating
3 Feb 2012