A few separate calls on my attention in the last week or so have set me thinking about attempts to make comparisons between the standards achieved by students in various parts of the world – and in particular the UK government’s desire to peg standards here to those achieved by students in ‘high performing jurisdictions’ (HPJ) (this seems to mean those parts of the world whose students do well in the PISA tests).
Reform of GCSE Sciences in the UK
Here in the Science Education Group at the University of York we are working with OCR, the awarding body, to develop the fourth iteration of Twenty First Century Science course, to be examined for the first time in June 2018. All GCSE specifications must meet the criteria laid down by Ofqual; for this latest development the Department for Education (DfE) determined the content for the new GCSEs. The criteria can be found on the DfE website:
When these documents were published the Secretary of State said that they were intended to “…make these qualifications more ambitious, with greater stretch for the most able …..….. These changes will increase the rigour of qualifications, strengthening the respect in which they are held by employers and universities alike. Young people in England deserve world-class qualifications and a world-class education – and that is what our reforms will deliver.” (Gove, M. 2014)
So that is the ambition of this change – but how will we know if it is achieved? If the measure of success is a higher ranking in the PISA league tables, these new criteria do not seem to be driving in the right direction.
OECD states that “Rather than examine mastery of specific school curricula, PISA looks at students’ ability to apply what they learn in school to real-life situations.” (OECD, 2014) Although OECD do not release the papers used for the rankings test, they do release some sample questions such as those here.
Looking at some of these questions, it is clear that to be successful students need to be able to read about science and to interpret what they read; as well as science knowledge they need an understanding of how science works.
The new GCSE criteria for 2016 are more content-laden, with LESS emphasis on how science works than the previous versions we have worked with, and as a result the constructs to be examined by the new GCSEs will differ from those assessed by PISA; there is no evidence that the new GCSE criteria have been written with a consistent reference to any other international benchmarks either.
Setting the standards
Ofqual (Office of Qualifications and Examinations Regulation) have had the task of determining how standards will be set for the new GCSEs. Currently GCSEs are graded from G-A*; some people, including the former Secretary of State, have expressed concerns that too many students were getting the top grade of A* (this has been ascribed to ‘grade inflation’, rather than the result of better teaching and hard work by students). The new grading system will run from 1-9, presumably allowing for the opportunity to add in a grade 10 if ‘inflation’ takes over again.
In a letter to Ofqual the Secretary of State wrote, “At the level of what is widely considered to be a pass (currently indicated by a grade C), there must be an increase in demand, to reflect that of high-performing jurisdictions.” (quoted by Ofqual in their recent report). However this is easier said than done. TIMMS (Trends in International Mathematics and Science Study) tests are carried out with students at an earlier stage in their schooling (grades 4 and 8); PISA science tests, taken by 15 year olds, do not measure exactly the same constructs as GCSE Sciences.
In September 2014 Ofqual published the report mentioned above in which they set out how the new grades would be determined. Essentially the new numbering system will be tied to the statistical measures currently used, with a more differentiated top end. The current grade C will become the new grade 4 and Grade 5 will become the new ‘pass’ mark.
There will be some post hoc checking to see whether their estimate of where to pitch the new ‘pass’ is right:
“49. The DfE analysis indicates that if students presently achieving grade Cs were to achieve grade 5s, that is broadly in line with what would be required to match the average performance of 16 year olds in England with the PISA mathematics performances of countries such as Finland, Canada, the Netherlands and Switzerland. Once the new grades have been awarded, we propose carrying out analyses to see how students with a grade 5 perform on international surveys such as PISA. We can then consider, in the context of Ofqual’s qualifications standards objective, whether our expectations of the grade 5 standard are being met. (Ofqual, 2014. page 11)”
So the new grading will be linked to the current system, to ensure continuity but there is no ‘setting of standards’, no descriptions of what a grade 5 means in terms of what a student can do.
What is happening elsewhere?
I have been aware for some time that there has been a project to write a new ‘national curriculum’ for science in the US. Each state, and indeed each district, has a lot of control over education but the Next Generation Science Standards (NGSS) have been developed in an attempt “to provide all students an internationally benchmarked science education” (sic) in the US. The process of writing these standards began with the development of A Framework for K-12 Science education, which was put together following much consultation within both the science community and the science education community. Altogether a much more measured approach than the recent developments of the National Curriculum here in England.
These Standards are now being used in schools and colleges across the US and the Twitter discussion on #iteachphysics this weekend was about standards-based grading. The discussion was led by @BlackPhysicists who asked me “Where can we get an articulation of the GSCE standards for physics that we can compare with NGSS & state stnds here?” This question drew me into think about whether it really is possible to make such comparisons.
In our current specifications (such as the Twenty First Science Physics specification) there are ‘Grade descriptors’ which were originally written to be used when setting grade boundaries after examination papers had been taken. But as Ofqual has said in their recent report:
“When GCSEs were first being developed in the mid-1980s the Government’s intention was that criteria-related grades would be introduced as soon as practicable with candidates who reached the required standard being awarded those grades. Despite heroic efforts, it proved impossible in practice to meet that intention. So GCSEs have never been criterion referenced.” (Ofqual, 2014. p5-6)
So the ‘standards’ are not laid down anywhere – the specifications describe the content to be learned and assessed, the Assessment Objectives describe the kinds of things candidates are expected to do in examinations, but nowhere is the expected standard described.
So how can we make comparisons?
It comes back to the same place that I always end up – it’s the questions we ask and the work of students in responding to those questions and tasks that show what they can do. So maybe that’s what we need to do – share questions and examples of student’s work.