Alex Harris, Assessment Standards Senior Manager and Andri Achilleos, Science Subject Advisor

Exams are over and as students, parents and teachers wait anxiously for the results, attention turns once again to grade boundaries. I spoke to Alex Harris from our assessment standards team to discuss how the awarding process works and how grade boundaries are set. 

In this blog Alex answers the most common questions that come up every year about grade boundaries.

Understanding comparable outcomes and awarding principles

What do we mean by comparable outcomes and why is it important? 

Comparable outcomes is the approach used by all awarding organisations when setting grade boundaries for general qualifications (GCSEs, AS and A Levels). The approach is agreed with Ofqual, the independent regulator of qualifications, examinations and assessments in England. 

It means that exam boards look at how the students performed in previous assessments (their prior attainment) and compare this with established historical trends. The trends show how students with similar levels of prior attainment have performed in the qualification. This is used to create predicted outcomes for the cohort being awarded for each qualification. In simple terms, it helps estimate what grades students with similar starting points would be expected to achieve. 

This is important because the difficulty of some assessments can vary from year to year. In addition, teachers and students become more familiar with qualifications and assessment requirements over time. To ensure fairness over time, we use predictions to show what grades we would reasonably expect the cohort to achieve, based on their prior attainment, regardless of how well everyone knows the course by that stage or what may have happened in a specific assessment. 

Using comparable outcomes allows exam boards to: 

• adjust grade boundaries up or down if an assessment was found to be more or less demanding than intended. 
• avoid unfair advantage or disadvantage caused by changes in assessment difficulty or by increasing familiarity 
• ensure results are fair and consistent over time. 

In practical terms, how do comparable outcomes prevent grade inflation/deflation? 

When we set grade boundaries, we’re trying to work out what’s changed from previous years and what’s the same. If results look similar to past cohorts, setting boundaries is more straightforward. However, sometimes the scores achieved for a cohort look quite different to previous years. We need to understand why. Is it due to changes in the assessment, a stronger or weaker cohort taking the qualification, growing familiarity with the assessment, or a mix of factors? 

Comparable outcomes give us the best statistical basis to help us make these judgements. It helps us decide if a cohort is showing genuine signs of improvement in terms of numbers achieving some grades, or whether other issues are driving changes in outcomes. 

This approach helps maintain fairness over time. That means looking not only at the present but also at the past. If results are allowed to gradually increase year-on-year because students and teachers become more familiar with the qualification and have access to more resources, it would lead to grade inflation. This would be unfair on previous cohorts who did not have those advantages. 

What forms of evidence (statistical predictions, prior attainment, cohort profiles) are used during awarding? 

It depends on the type of qualification and award, but in general we look at: 

• data about how each assessment performed such as the mean mark, standard deviation, the distribution of marks and percentage outcomes across the mark range. For examined assessments we often have data on different questions and/or sections of the assessment as well.
• data on prior attainment as part of the predictions and age and cohort profile in terms of centres entering. 
• feedback from the lead assessors on each assessment and the assessment teams. 
• judgments from a panel of senior assessors on evidence (such as exam scripts or pieces of NEA work) for a range of marks relevant to certain grades (judgmental grades). 
• feedback from centres and any issues with the assessment (such as if there was an error in a question or we had complaints about aspects of an assessment).

Cohort performance and prediction

Do awarding bodies hold a fixed number or percentage of students that must achieve each grade?

Not a fixed percentage, but where comparable outcomes are used then the predictions do suggest what proportion of students should achieve each grade. There is some flexibility in terms of meeting these predictions, but they provide a fairly strong indication of the outcomes we would normally expect. 

Face validity and public confidence is also important. We have to consider the plausibility of outcomes being radically different from one year to another, and both the fairness and the perception of this.

Do exam boards set a target grade boundary (such as the A or E, Grade 4 or 9 boundary) before writing the exam paper?

Yes. Assessments such as exam papers are usually written with design thresholds in mind. Design thresholds are basically ideal grade boundaries which would give a good spread and distribution of marks and differentiate students effectively, assuming that the assessment works broadly as intended. 

They tend to be spread at equal increments across the mark range. For AS and A Levels we typically aim for 80% for an A grade and 40% for an E grade. Grades in between would be 10% apart, for example 60% for a C. When writing the assessments, authors are asked to say how the marks in the questions they are setting link to the 80% and 40% targets.

What checks are in place to ensure that the paper as designed sits at the intended level of demand? 

Comparable outcomes and predictions help with this because they give us some indication of cohort ability independent of how an assessment performed that series and other factors which may have influenced outcomes. Professional judgment also plays an important role. Awarding panels look at students’ work alongside an archive of material from previous series, at different grade boundaries. This helps us check that what we award each time looks and feels comparable to the established performance standards.

New qualifications, including AAQs

What types of interim guidance or professional judgement indicators can be shared to support centres before the first cohort sits the exam? 

The design thresholds mentioned earlier provide the best guide to likely grade boundaries before an assessment has been sat. They give teachers and students a reasonable guide when using Sample Assessment Materials (SAMs) or practice papers. 

But while we can often set the live assessment grade boundaries on or near these design thresholds, this is not always the case. A range of factors are considered in setting the grade boundaries for live assessment and we can sometimes deviate some way away from them when this is appropriate. 

How are first year awards set when there is no previous exam data to use for comparable outcomes? 

This can depend on the qualification in question. In general, it’s a balance between continuity of standards established in relevant previous qualifications alongside establishing a ‘new’ standard for the new qualification. There may be significant changes in design or structure as a result of Department for Education (DfE) or Ofqual directives. 

The qualification type and model is also considered. For example, for a modular qualification with January and June assessments where students can re-sit units, we would look at how the first January assessment seems to have performed against another comparable assessment in an established qualification. But we would not necessarily expect the same level of performance or similar outcomes in the new qualification after one term of teaching the new course. A lower level of performance may well reflect the level of preparedness of the cohort. 

For further guidance on grading and comparable outcomes you can explore Ofqual’s published resources, as well as the our marking and grading factsheet. If you would like to discuss this further or have specific questions about awarding, please feel free to contact us. We are always happy to support teachers and centres in understanding how grading decisions are made.

Stay connected

If you have any questions, you can email us at support@ocr.org.uk or call us on 01223 553998. You can also sign up to subject emails to keep up-to-date with the latest news, information and resources.

If you are considering teaching any of our qualifications, use our online form to let us know, so that we can help you with more information.

About the author

Andri Achilleos was a teacher for ten years before joining Cambridge OCR in January 2019 as the subject advisor for A Level Biology. She studied Biology at University of Bristol and completed an MA in Science Education at University of York. She has taught in Birmingham as Teacher in charge of Biology, as well as an international school in Europe. During her teaching career she has taken on various roles within the department and has also been an examiner for different exam boards.

Alex Harris has worked for Cambridge OCR since 2002, initially in a variety of administration and operations roles, then as a Qualifications Manager from 2005 to 2012, working predominantly on PE and Sport qualifications. He has worked in Assessment Standards since February 2012 across a wide range of subjects and qualifications and specialises in vocational qualifications and performance-based NEA. Alex studied Public Policy, Government and Management at the University of Birmingham from 1998 to 2001.