Software

You're testing them wrong: Whiteboard coding interviews are 'anti-women psychological stress examinations'

Boffins find flaws in the way software engineering hopefuls are sifted


People applying for software engineering positions at companies are often asked to solve problems on a whiteboard, under the watchful eye of an interviewer, as a way to assess technical problem solving skills.

But recent research suggests that whiteboard technical tests – so daunting to job seekers that there are books on how to deal with them – often fail to assess technical skill, according to new research. Instead, they're all about pressure.

In a paper [PDF] to be presented later this year at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, researchers from North Carolina State University (NCSU) and Microsoft in the US argue that whiteboard sessions test for stage fright rather than, y'know, coding competency.

The title of the paper hints at its conclusion: "Does Stress Impact Technical Interview Performance?" NCSU authors Mahnaz Behroozi, Shivani Shirolkar, and Chris Parnin, with Titus Barik from Microsoft, say it most certainly does.

"Through a happy accident, the software industry has seemingly reinvented a crude yet effective instrument for reliably introducing stress in subjects, which typically manifests as performance anxiety," the paper explains.

When Facebook says you're not a good 'culture fit', it means you're not White or Asian enough – complaint

READ MORE

"A technical interview has an uncanny resemblance to the Trier Social Stress Test, a procedure used for decades by psychologists and is the best known 'gold standard' procedure for the sole purpose of reliably inducing stress."

As a consequence, whiteboard interviews may fail to assess coder competency. Rather, the researchers argue, they measure how well job candidates handle anxiety.

Using 48 graduate and undergraduate students with programming experience, the researchers conducted a randomized controlled trial to compare the traditional technical interview (done while being watched) with a private session evaluation (done without being observed). The experiment was designed to measure cognitive load and stress through the collection of eye tracking metrics, specifically fixation duration and pupil dilation.

The researchers found that stress hinders interview performance, with participants in the traditional technical interview exhibiting higher cognitive load, lower scores, and higher stress levels. In essence, social anxiety took otherwise qualified job candidates out of the running because of the circumstances of the interview.

Further flaws

What's more, whiteboard technical interviews appear to favor men over women.

"We also observed that no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting," the paper says.

In a phone interview with The Register, Christopher Parnin, assistant professor at NC State University and one of the paper's co-authors, said he doesn't have a conclusive reason why this might be the case. He said there's some support in academic literature to indicate the women have more performance anxiety than men, but he stressed that's a gross oversimplification because men experience performance anxiety too.

For Parnin, the problem is whiteboard tests themselves. "It all comes down to the fact that the test is designed to make almost anyone fail," he said. "You're basically having to interview tons of people just to find those who can pass it."

Parnin took issue with the way the industry has dealt with the difficulty of its tests. Rather than coming up with a fair way to evaluate software engineers, companies like Google advise at least 40 practice sessions - a time commitment that's not an option for everyone. This amounts to stress inoculation training and it does help people pass whiteboard tests, he said, but it doesn't make the tests an effective skill assessment tool.

As an alternative, the paper points to the way devops biz Honeycomb (Hound Technology) – overseen by a female CEO, CTO, CMO and VP of engineering – approaches hiring. The company provides interview questions in advance so it's not a Trier Social Stress Test.

As the company explains on its website, its goal is to avoid surprises. "The research is clear: unknowns cause anxiety, and people don’t perform well when they’re anxious," the company says.

"The big picture is to provide more accessible alternatives," said Parnin. "There are a lot of ways to test for the same thing without putting all this pressure on people." ®

Send us news
123 Comments

Industrial robots make people feel worse about jobs and themselves

Study finds workers' sense of meaningfulness and autonomy declines with automation

Tough luck, bosses, AI is coming for your job, too

Algorithms as PHBs – who wouldn't want that?

Google Cloud chief is really psyched about this AI thing

We're on a highway to ML

VMS Software prunes OpenVMS hobbyist program

Vintage OS editions go the way of the dodo as VSI cranks up exclusivity

What if AI produces code not just quickly but also, dunno, securely, DARPA wonders

As 70% of boffinry nerve center's projects involve machine learning

Chrome Enterprise Premium promises extra security – for a fee

Paying for browsers is no longer a memory from the 1990s

Sleuths who cracked Zodiac Killer's cipher thank the crowd

Fifty-one years of community contributions, software, and clever cryptanalysis contributed

How to coax ChatGPT into making better predictions: Get it to tell tales from the future

'Something is stopping it, even though it clearly can do it'

OpenBSD 7.5 locks down with improved disk encryption support and syscall limitations

The most secure Unix-like OS to date?

British watchdog has 'real concerns' about the staggering love-in between cloud giants and AI upstarts

Billions in investment? Yeeeah, right – looks more like ensuring only select few developers thrive

Boffins deem Google DeepMind's material discoveries rather shallow

Web titan rejects criticisms, insists AI-found compounds are legit

Rust developers at Google are twice as productive as C++ teams

Code shines up nicely in production, says Chocolate Factory's Bergstrom