Pass rates are at 98 per cent. A quarter of grades are an A or higher. This week, every newspaper in the country was filled with people asserting that exams are definitely getting easier, and other people asserting that exams are definitely not getting easier. The question is always simple: how do you know?
Firstly, the idea of kids getting cleverer is not ludicrous. ‘The Flynn Effect’ is a term coined to describe the gradual improvement in IQ scores. This has been an important problem for IQ researchers, since IQ tests are peer referenced: that is, your performance is compared against everyone else, and the scores are rejigged so that the average IQ is always 100. Because of the trend to higher scores, year on year, you have to be careful not to use older tests on current populations, or their scores come out spuriously high, by the standards of the weaker average population of the past. Regardless of what you think about IQ tests, the tasks in them are at least relatively consistent. That said, there’s also some evidence that the Flynn effect has slowed in developed countries recently.
But ideally, we want research that addresses exams directly. One approach would be to measure current kids’ performance on the exams of the past. This is what the Royal Society of Chemistry did in its report ‘The Five Decade Challenge’ in 2008, running the project as a competition for sixteen-year-olds, which netted them 1,300 self-selecting higher-ability kids. They sat tests taken from the numerical and analytical components of O-level and GCSE exams over the past half-century, and it was found that performance against each decade rose over time: the average score for the 1960s questions was 15 per cent, rising to 35 per cent for the current exams (though with a giant leap around the introduction of GCSEs, after which the score remained fairly stable).
There are often many possible explanations for a finding. These results could mean that exams have got easier, but it’s also possible that syllabuses have changed, so modern kids are less prepared for older-style questions. When the researchers looked at specific questions, they found that some things had been removed from the GCSE syllabus – because they’d moved up to A-level – but that’s drifting unwelcomely towards anecdote.
Another approach would be to compare performance on a consistent test, over the years, against performance on A-levels. Robert Coe at Durham University produced a study of just this for the Office of National Statistics in 2007. Every year since 1988 a few thousand children have been given the Test of Developed Abilities, a consistent test (with a blip in 2002) of general maths and verbal reasoning skills. The scores saw a modest decline over the 1990s, and have been fairly flat for the past decade. But the clever thing is what the researchers did next: they worked out the A-level scores for children, accounting for their TDA scores, and found that children with the same TDA score were getting higher and higher exam results. From 1988 to 2006, for the same TDA score, A-level results rose by an average of two grades in each subject.
It could be that exams are easier. It could be that teaching and learning have improved, or that teaching has become more exam-focused, so kids at the same TDA level do better in A-levels: this is hard to measure. It could be that TDA scores are as irrelevant as shoe size, so the finding is spurious.
Alternatively, it could be that exams are different: they might be easier, say, with respect to verbal reasoning and maths, but harder with respect to something else. This, again, is hard to quantify. If the content and goals of exams change, then that poses difficulties for measuring their consistency over time, and it might be something to declare loudly (or to consult employers and the public about, since they seem concerned).
Our last study thinks more clearly along those lines: some people do have clear goals from education, and they can measure students against this yardstick, over time. ‘Measuring the Mathematics Problem’ is a report done for the Engineering Council and other august bodies in 2000, analysing data from sixty university maths, physics and engineering departments which gave diagnostic tests on basic maths skills to their new undergraduates each year. These were tests on things that mattered to university educators, and if something educational matters to them, we might think it matters overall. They found strong evidence of a steady decline in scores on their own tests, over the preceding decade, among students accepted onto degree courses where they would need good maths skills.
Sadly they didn’t control for A-level grade, so we can’t be sure how far they were comparing like with like, but there are various plausible explanations for their finding. Maybe maths syllabuses changed, and were less useful for maths and engineering degrees. Maybe the cleverest kids are doing English these days, or becoming lawyers instead. Or maybe exams got easier.
If you know of more research, I’d be interested to see it, but the main thing that strikes me here is the paucity of work in the field. There’s a man called Rupert Sheldrake who believes that pets are psychic: they know when their owners are coming home, that sort of thing. Obviously we disagree on a lot, but we chat, and are friendly, and once when we were talking he came out with an excellent suggestion: maybe 1 per cent – or even 0.01 per cent – of the total UK research budget could be given to the public, so that they could decide what their research obsessions were. Maybe most of this money would get spent on psychic pets, or research into which vegetables cure cancer, but since we’re all clearly preoccupied with the idea, I’d like to think that some of it, possibly, might get spent on good-quality, robust research to find out whether exams are getting easier.