Software

You're testing them wrong: Whiteboard coding interviews are 'anti-women psychological stress examinations'

Boffins find flaws in the way software engineering hopefuls are sifted


People applying for software engineering positions at companies are often asked to solve problems on a whiteboard, under the watchful eye of an interviewer, as a way to assess technical problem solving skills.

But recent research suggests that whiteboard technical tests – so daunting to job seekers that there are books on how to deal with them – often fail to assess technical skill, according to new research. Instead, they're all about pressure.

In a paper [PDF] to be presented later this year at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, researchers from North Carolina State University (NCSU) and Microsoft in the US argue that whiteboard sessions test for stage fright rather than, y'know, coding competency.

The title of the paper hints at its conclusion: "Does Stress Impact Technical Interview Performance?" NCSU authors Mahnaz Behroozi, Shivani Shirolkar, and Chris Parnin, with Titus Barik from Microsoft, say it most certainly does.

"Through a happy accident, the software industry has seemingly reinvented a crude yet effective instrument for reliably introducing stress in subjects, which typically manifests as performance anxiety," the paper explains.

When Facebook says you're not a good 'culture fit', it means you're not White or Asian enough – complaint

READ MORE

"A technical interview has an uncanny resemblance to the Trier Social Stress Test, a procedure used for decades by psychologists and is the best known 'gold standard' procedure for the sole purpose of reliably inducing stress."

As a consequence, whiteboard interviews may fail to assess coder competency. Rather, the researchers argue, they measure how well job candidates handle anxiety.

Using 48 graduate and undergraduate students with programming experience, the researchers conducted a randomized controlled trial to compare the traditional technical interview (done while being watched) with a private session evaluation (done without being observed). The experiment was designed to measure cognitive load and stress through the collection of eye tracking metrics, specifically fixation duration and pupil dilation.

The researchers found that stress hinders interview performance, with participants in the traditional technical interview exhibiting higher cognitive load, lower scores, and higher stress levels. In essence, social anxiety took otherwise qualified job candidates out of the running because of the circumstances of the interview.

Further flaws

What's more, whiteboard technical interviews appear to favor men over women.

"We also observed that no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting," the paper says.

In a phone interview with The Register, Christopher Parnin, assistant professor at NC State University and one of the paper's co-authors, said he doesn't have a conclusive reason why this might be the case. He said there's some support in academic literature to indicate the women have more performance anxiety than men, but he stressed that's a gross oversimplification because men experience performance anxiety too.

For Parnin, the problem is whiteboard tests themselves. "It all comes down to the fact that the test is designed to make almost anyone fail," he said. "You're basically having to interview tons of people just to find those who can pass it."

Parnin took issue with the way the industry has dealt with the difficulty of its tests. Rather than coming up with a fair way to evaluate software engineers, companies like Google advise at least 40 practice sessions - a time commitment that's not an option for everyone. This amounts to stress inoculation training and it does help people pass whiteboard tests, he said, but it doesn't make the tests an effective skill assessment tool.

As an alternative, the paper points to the way devops biz Honeycomb (Hound Technology) – overseen by a female CEO, CTO, CMO and VP of engineering – approaches hiring. The company provides interview questions in advance so it's not a Trier Social Stress Test.

As the company explains on its website, its goal is to avoid surprises. "The research is clear: unknowns cause anxiety, and people don’t perform well when they’re anxious," the company says.

"The big picture is to provide more accessible alternatives," said Parnin. "There are a lot of ways to test for the same thing without putting all this pressure on people." ®

Send us news
123 Comments

Senate dishes out second helping of SAMOSA to kill costly software licenses

House version of software license management bill introduced in March has yet to budge amid distractions

US auditors beg Pentagon to pay attention to latest report about IT system flaws

The DoD has ignored many past recommendations

Linux Foundation tries to play peacemaker in ongoing WordPress scuffle

FAIR Package Manager project aims to prevent political power plays

Apple goes glass whole as it pours new UI everywhere

Annual infomercial spruiks visual change, modest functional enhancements, and a movie

Enterprises are getting stuck in AI pilot hell, say Chatterbox Labs execs

Security, not model performance, is what's stalling adoption

Boffins found self-improving AI sometimes cheated

Instead of addressing hallucinations, it just bypassed the function they built to detect them

Tesla FSD ignores school bus lights and hits 'child' dummy in staged demo

Its driverless cars are already testing in Austin - good luck

AI coding tools are like that helpful but untrustworthy friend, devs say

A survey from AI biz Qodo finds robo-coding productivity gains are unevenly distributed

Anthropic CEO frets about 20% unemployment from AI, but economists are doubtful

It's hard to predict the future, but trying might get Amodei a seat at the regulatory table

IBM Watson zombie brand shuffles forward with new AI lab in NYC

Unsurprisingly, it's all about agents, the buzzword du jour

AI kept 15-year-old zombie vuln alive, but its time is drawing near

Researchers have come up with a fix for a path traversal bug first spotted in 2010

Meta offered one AI researcher at least $10,000,000 to join up

Mark Zuckerberg reached out to our source directly