Why did Johnny and Jenny's exam grades yo-yo over the summer? Here's some of the code behind UK results chaos

Ofqual publishes for the sake of 'transparency' but it's clear as mud


UK exams body Ofqual has published the code behind the summer's results fiasco. Sort of.

Like a 1980s hobbyist keenly keying in source from Home Computing Weekly and hunting for that sneaky bug or typo, you too can now peer at reams of code and wonder just how it all went so wrong. Assuming the statistical language R is your thing.

Presented in GitHub, the aim is to promote transparency. A laudable goal, if it weren't for the fact that, as the repo makes clear, "the code developed and presented here was not the final code used." It was up to exam boards to determine that. All Ofqual did was flash the source to demonstrate "how Ofqual's relevant regulatory requirements could be implemented."

Another fly in the ointment is a lack of data to run through the algorithms in order to enjoy a repeat of the summer's fireworks from the comfort of one's own computer. Ofqual is working with Ofsted, the Department for Education, and UCAS to make available the data used in 2020, but for now one will have to build one's own .csv files.

Even so, the R code is relatively easy to follow, with the key steps that built the historical performance and generated the predictions clearly visible in the source.

As a reminder, summer's fiasco was a result of the UK government opting to cancel the summer 2020 GCSE, AS, and A-level exam series in light of the COVID-19 pandemic. However, students still needed grades to progress to higher education, work, and so forth. Teachers were instead asked to submit the grades they reckoned their students would have achieved in the form of a Centre Assessment Grade (CAG).

Because teachers might be prone to optimism, those CAGs were standardised using Ofqual's preferred model, the Direct Centre Performance model (DCP), which would adjust things based on the historical performance of the college.

Those initialisms are important, and crop up in the code quite a bit.

Infamously, those calculated grades (which sparked protests after a good portion of students were downgraded) were abandoned and the teacher-predicted results used instead.

Guy Nason, chair of statistics at Imperial College London,welcomed the publishing of the source, declaring: "It's really good they've done this; we're on the way to transparency," although he noted that it would have been interesting to see what was in those "final codes". He also pointed out that communication in and out of the code "appears to be via spreadsheets" with a suitably scowly emoji.

As well as seeking some realistic sample data, Nason made a few of what he called "nerd points", observing it was "weird that they've essentially redefined the round function from R. Why? At first sight, seems to be written correctly, but why do it?" He added: "There's an interesting line in the DCP modelling code, where marks are removed from the cut point calculation where candidates had more than one mark from a previous year."

"It makes me wonder what else is in the code that isn't described in the written spec?"

While Ofqual is keen that this should be taken as a learning experience, the COVID-19 pandemic is continuing to disrupt education in the UK (and around the globe). As such, a look at what informed the thinking of 2020 would be useful ahead of whatever happens in 2021. ®


Tech Resources

Apps are Essential, so your WAF must be effective

You can’t run a business today without applications—and because apps are critical to strategic business imperatives and commerce, they have become the prime target for attackers.

Three reasons you need a hybrid multicloud

Businesses need their IT teams to operate applications and data in a hybrid environment spanning on-premises private and public clouds. But this poses many challenges, such as managing complex networking, re-architecting applications for the cloud, and managing multiple infrastructure silos. There is a pressing need for a single platform that addresses these challenges - a hybrid multicloud built for the digital innovation era. Just this Regcast to find out: Why hybrid multicloud is the ideal path to accelerate cloud migration.

How backup modernization changes the ransomware game

If the thrill of backing up your data and wondering if you will ever see it again has worn off, start the new year by getting rid of the lingering pain of legacy backup. Bipul Sinha, CEO of the Cloud Data Management Company, Rubrik, and Miguel Zatarain, Director of Global Infrastructure Technology at PACCAR, Fortune 500 manufacturer of trucks and Rubrik customer, are talking to the Reg’s Tim Phillips about how to eliminate the costly, slow and spotty performance of legacy backup, and how to modernize your implementation in 2021 to make your business more resilient.

Top 20 Private Cloud Questions Answered

Download this asset for straight answers to your top private cloud questions.

Biting the hand that feeds IT © 1998–2021