Commentary: What if the HR department's AI prefers resumes written by AI?

New: You can now listen to articles.

This audio is generated by an AI tool.

LONDON: Earlier this year, in the depths of summer, the Addleshaw Goddard law firm found itself in a slight pickle. In what was becoming a familiar pattern, an advert it had placed for an IT job in its human resources department had drawn a slew of applications.

A year ago, only a couple of dozen people might have sent in a resume for a position like this one. But AI has made the job of applying for a job much faster and easier, and this time 178 applications arrived.

The problem was, the longer it took to sift them, the higher the chance that good candidates would find work elsewhere. Like a lot of organisations, Addleshaw has been encouraging its managers to test ways that AI can safely improve its processes, so the firm’s HR department decided to do an experiment. 

It would see how well an AI agent could assess the applications compared with humans.

AI AND HUMANS PREFERRED DIFFERENT APPLICATIONS

An agent was duly set up, using the firm’s internal Microsoft Copilot system, and told about the role that needed to be filled, as well as the experience applicants would need with certain HR systems and programming languages, and with AI itself.

Humans at the firm sorted the applicants into a shorter pile of about a dozen and asked the agent to score them. 

The agent whipped through this task and presented a list of what it deemed the top contenders. The humans did the same thing.

Alas, the two lists did not match. The agent was impressed by glossy applications from candidates who claimed to have achieved exceptional IT feats in record time at a previous employer, or worked on big projects despite scant levels of experience. In short, they were too good to be true – and smacked of having been written with the help of AI.

Katherine Rathbone, Addleshaw Goddard’s head of HR, told me her team had theories about why this might have happened.

For one thing, AI tools can make it a lot easier for candidates to tailor a CV to match a job description. But that was not all. “One possible explanation is that, because AI capability was itself one of the criteria, the agent may have interpreted signs of AI use in producing the CV as evidence of AI capability,” she said.

USING THE SAME AI AS THE RECRUITER?

There is another, more disturbing possibility. The AI agent may have actively preferred CVs purely because AI was involved in writing them. 

If so, it would confirm the findings of an eye-catching working paper that has analysed the behaviour of AI hiring tools.

The researchers simulated what they said were realistic hiring processes for 24 jobs ranging from teachers and consultants to chefs and engineers. They discovered that candidates using the same AI large language model as the recruiter were between 23 per cent and 60 per cent more likely to be shortlisted than those who relied on CVs produced by a hapless human.

This human handicap was highest for business roles such as sales and accountancy and lowest for fields such as farming and automobiles.

The paper is still being revised for publication in a peer-reviewed journal, meaning some of its findings may change. But one of its authors, Jiannan Xu, told me that updated analyses continue to provide evidence that AI models “exhibit self-preferencing behaviour”.

DISTURBING NEWS IN THE HIRING ARMS RACE

That is disturbing news at a time when AI is already driving a dismal hiring arms race. On one side are job hopefuls who, convinced they are battling a wall of soulless recruitment tools, are fighting back with an explosion of AI applications that are swamping recruiters.  

If AI is biased towards AI, it adds a fresh dimension of jeopardy. Imagine being the candidate using the wrong type of AI, or no AI at all.

Fortunately, Xu’s paper suggests AI bias can be more than halved with two relatively simple strategies: telling models to ignore the origin of resumes and focus on their content, and using a number of assessment models to dilute the bias of any single tool.

Meanwhile at Addleshaw Goddard, I was pleased to learn that the successful candidate for the IT job was chosen by humans, and was not the one the agent ranked highest.

And as for using AI for recruitment, it is still a work in progress. The tools clearly have potential if used correctly, says Rathbone. “But human judgment remains important.”