By Avi Asher-Schapiro
NEW YORK, Aug 12 (Thomson Reuters Foundation) - As millions
of students in Britain await exam results this month, a row has
erupted over algorithms used to calculate their grades after
exams were cancelled due to the new coronavirus.
In a last-minute change following an outcry in Scotland,
education minister Gavin Williamson said on Wednesday that
students set to receive their A level marks - used to enter
university - on Thursday could use the results of earlier tests.
The credibility of new grading algorithms has come into
doubt not just in Britain's national exams, but also for the
International Baccalaureate (IB) programme used in more than 150
countries.
HOW DOES A GRADING ALGORITHM WORK?
In Britain, teachers were asked to submit to exam boards the
grades they thought each student would have achieved, and to
rank their pupils in order.
These data points for each student were then put through a
statistical model, including the historic performance of their
school, to determine whether the teachers' grades were more
severe or generous than expected and matched previous years.
WHAT'S WRONG WITH THAT?
Relying on algorithms to determine results has the potential
to reproduce - or even exacerbate - existing patterns of
inequality for low-income and minority students, according to
some education researchers and statisticians.
Critics say poorer students risk being unfairly penalised
because they are judged on the track record of their schools
rather than individual performance.
WHY THE UPROAR IN SCOTLAND?
Scotland on Tuesday promised to restore 75,000 grades that
were downgraded by its moderation system to the original levels
set by teachers.
The algorithm had reduced the pass rate for Highers - which
18-year-olds use to gain a place at university - by 15% for
pupils from the most deprived backgrounds, while those for the
wealthiest pupils only fell by 7%.
"We believe we have delivered fairness to learners, through
a consistent, evidence-based approach in the absence of exams,"
a spokesman for the Scottish Qualifications Authority, which
oversees the process, said in emailed comments.
CAN ALGORITHMS LEAD TO BIAS?
There was also an outcry in July when results for the IB -
sat by 160,000 students globally each year - were released.
Most IB students live in the United States, where critics
said the algorithm could lead to bias against minorities and low
income students.
Almost 60% of public schools offering IBs are "Title I"
schools which have significant low-income populations, according
to the programme.
The IB model took into account historic data from each
school, raising questions about whether students in poor
communities, with less resources, might be unfairly downgraded.
"Racial, gender, socioeconomic or any other sort of bias has
no place at the IB," a spokesman said in a statement.
"We continue to work closely with the regulators and experts
that helped shape and approve the models that assessed student
results."
CAN STUDENTS CHALLENGE THEIR RESULTS?
In Norway, IB students complained to the local data
protection authority (DPA), which enforces laws requiring
Norwegians' personal data - including grades - be processed
fairly, accurately and transparently.
One student, Sigurd Salvesen, told the Thomson Reuters
Foundation that he was assigned an IB grade of 30 out of 45,
which fell short of the 34 he needed to take up an offer from
the University of Edinburgh in Scotland.
"I don't feel like I was graded based on my own work or
achievements," he said, adding that he feared the algorithm
assigned him a low grade because of his school's past poor
performance.
In a draft decision, the DPA said on Friday it planned to
order the IB Organisation to issue fresh grades without
reference to "school context" or "historical data" as these had
no connection to individual academic achievements.
"The name of the school or its geographical location is not
a relevant element in awarding grades," it said.
"This may have discriminatory effects as the model can
potentially be of prejudice to certain socio-economic groups ...
students from not-so-well-performing areas would be penalised
just for going to school in that area, which is clearly unfair."
The DPA has given the IB Organisation until Aug. 21 to
submit its response before issuing a final decision.
WHAT HAPPENS NEXT?
IB students in Britain are also hoping for new grades after
the exam watchdog, Ofqual, asked the IB Organisation for more
information about its grading process.
Meanwhile, A level results for England, Wales and Northern
Ireland will be released on Thursday, followed by GCSE results
for 16-year-olds on Aug. 20.
Britain's Royal Statistical Society, a professional body,
has called for a "full and open appraisal" of the methods used,
suggesting it would be fairer to include individual students'
historic performance rather than that of their school.
"If the underlying process is flawed then, possibly the
whole thing is flawed," said Guy Nason, chairman of the Royal
Statistical Society's research section, adding that it was hard
to judge the method as it still has not been made public.
(Reporting by Avi Asher-Schapiro @AASchapiro, Editing by Katy
Migiro. Please credit the Thomson Reuters Foundation, the
charitable arm of Thomson Reuters, that covers the lives of
people around the world who struggle to live freely or fairly.
Visit http://news.trust.org)