You're testing them wrong: Whiteboard coding interviews are 'anti-women psychological stress examinations'

Boffins find flaws in the way software engineering hopefuls are sifted

People applying for software engineering positions at companies are often asked to solve problems on a whiteboard, under the watchful eye of an interviewer, as a way to assess technical problem solving skills.

But recent research suggests that whiteboard technical tests – so daunting to job seekers that there are books on how to deal with them – often fail to assess technical skill, according to new research. Instead, they're all about pressure.

In a paper [PDF] to be presented later this year at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, researchers from North Carolina State University (NCSU) and Microsoft in the US argue that whiteboard sessions test for stage fright rather than, y'know, coding competency.

The title of the paper hints at its conclusion: "Does Stress Impact Technical Interview Performance?" NCSU authors Mahnaz Behroozi, Shivani Shirolkar, and Chris Parnin, with Titus Barik from Microsoft, say it most certainly does.

"Through a happy accident, the software industry has seemingly reinvented a crude yet effective instrument for reliably introducing stress in subjects, which typically manifests as performance anxiety," the paper explains.

When Facebook says you're not a good 'culture fit', it means you're not White or Asian enough – complaint


"A technical interview has an uncanny resemblance to the Trier Social Stress Test, a procedure used for decades by psychologists and is the best known 'gold standard' procedure for the sole purpose of reliably inducing stress."

As a consequence, whiteboard interviews may fail to assess coder competency. Rather, the researchers argue, they measure how well job candidates handle anxiety.

Using 48 graduate and undergraduate students with programming experience, the researchers conducted a randomized controlled trial to compare the traditional technical interview (done while being watched) with a private session evaluation (done without being observed). The experiment was designed to measure cognitive load and stress through the collection of eye tracking metrics, specifically fixation duration and pupil dilation.

The researchers found that stress hinders interview performance, with participants in the traditional technical interview exhibiting higher cognitive load, lower scores, and higher stress levels. In essence, social anxiety took otherwise qualified job candidates out of the running because of the circumstances of the interview.

Further flaws

What's more, whiteboard technical interviews appear to favor men over women.

"We also observed that no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting," the paper says.

In a phone interview with The Register, Christopher Parnin, assistant professor at NC State University and one of the paper's co-authors, said he doesn't have a conclusive reason why this might be the case. He said there's some support in academic literature to indicate the women have more performance anxiety than men, but he stressed that's a gross oversimplification because men experience performance anxiety too.

For Parnin, the problem is whiteboard tests themselves. "It all comes down to the fact that the test is designed to make almost anyone fail," he said. "You're basically having to interview tons of people just to find those who can pass it."

Parnin took issue with the way the industry has dealt with the difficulty of its tests. Rather than coming up with a fair way to evaluate software engineers, companies like Google advise at least 40 practice sessions - a time commitment that's not an option for everyone. This amounts to stress inoculation training and it does help people pass whiteboard tests, he said, but it doesn't make the tests an effective skill assessment tool.

As an alternative, the paper points to the way devops biz Honeycomb (Hound Technology) – overseen by a female CEO, CTO, CMO and VP of engineering – approaches hiring. The company provides interview questions in advance so it's not a Trier Social Stress Test.

As the company explains on its website, its goal is to avoid surprises. "The research is clear: unknowns cause anxiety, and people don’t perform well when they’re anxious," the company says.

"The big picture is to provide more accessible alternatives," said Parnin. "There are a lot of ways to test for the same thing without putting all this pressure on people." ®

Send us news

You thought you bought software – all you bought was a lie

Isn't code something worth paying for? Well, frankly, no

Atlassian smartens up security, licensing admin tools

And bundles its best bits at bargain price in case someone wants you to work their way for a while

Oracle verifies Java licensing tools from Flexera and Lime Software

While one vendor says they help prepare for audit, expert warns they don't defend compliance

NetScaler reclaims identity after Citrix, Tibco merge as ‘Cloud Software Group’

Other business units also made full members of group, perhaps hinting and spin-outs

Matrix chat encryption sunk by five now-patched holes

You take the green pill, you'll spend six hours in a 'don't roll your own crypto' debate

OpenAI opens doors to DALL-E after the horse has bolted to Midjourney and others

Ironic that an ML lab with so many accelerators is such a slowpoke

Cloudflare's invisible CAPTCHA works by probing browsers with JavaScript

Beta-grade widget respects your privacy, we're promised

How one Ukrainian software maker planned for survival as invaders approached

Set priorities, expect confusion, keep emergency instructions simple – that's just for starters

SAS puts ML and analytics suite on Azure Marketplace

50-year-old software project provides a button the boss can click

Another UK tech company bought out: Schneider Electric grabs rest of Aveva

French industrial conglomerate pays $10.75b, but already owned lion's share of industrial software specialist

IceWM reaches version 3 after a mere 25 years

And it's not the only venerable window manager still in development

VideoLAN to India: If you love FOSS so much, why have you blocked our downloads?

Activists help pen letter to New Delhi demanding answers