Software

You're testing them wrong: Whiteboard coding interviews are 'anti-women psychological stress examinations'

Boffins find flaws in the way software engineering hopefuls are sifted


People applying for software engineering positions at companies are often asked to solve problems on a whiteboard, under the watchful eye of an interviewer, as a way to assess technical problem solving skills.

But recent research suggests that whiteboard technical tests – so daunting to job seekers that there are books on how to deal with them – often fail to assess technical skill, according to new research. Instead, they're all about pressure.

In a paper [PDF] to be presented later this year at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, researchers from North Carolina State University (NCSU) and Microsoft in the US argue that whiteboard sessions test for stage fright rather than, y'know, coding competency.

The title of the paper hints at its conclusion: "Does Stress Impact Technical Interview Performance?" NCSU authors Mahnaz Behroozi, Shivani Shirolkar, and Chris Parnin, with Titus Barik from Microsoft, say it most certainly does.

"Through a happy accident, the software industry has seemingly reinvented a crude yet effective instrument for reliably introducing stress in subjects, which typically manifests as performance anxiety," the paper explains.

When Facebook says you're not a good 'culture fit', it means you're not White or Asian enough – complaint

READ MORE

"A technical interview has an uncanny resemblance to the Trier Social Stress Test, a procedure used for decades by psychologists and is the best known 'gold standard' procedure for the sole purpose of reliably inducing stress."

As a consequence, whiteboard interviews may fail to assess coder competency. Rather, the researchers argue, they measure how well job candidates handle anxiety.

Using 48 graduate and undergraduate students with programming experience, the researchers conducted a randomized controlled trial to compare the traditional technical interview (done while being watched) with a private session evaluation (done without being observed). The experiment was designed to measure cognitive load and stress through the collection of eye tracking metrics, specifically fixation duration and pupil dilation.

The researchers found that stress hinders interview performance, with participants in the traditional technical interview exhibiting higher cognitive load, lower scores, and higher stress levels. In essence, social anxiety took otherwise qualified job candidates out of the running because of the circumstances of the interview.

Further flaws

What's more, whiteboard technical interviews appear to favor men over women.

"We also observed that no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting," the paper says.

In a phone interview with The Register, Christopher Parnin, assistant professor at NC State University and one of the paper's co-authors, said he doesn't have a conclusive reason why this might be the case. He said there's some support in academic literature to indicate the women have more performance anxiety than men, but he stressed that's a gross oversimplification because men experience performance anxiety too.

For Parnin, the problem is whiteboard tests themselves. "It all comes down to the fact that the test is designed to make almost anyone fail," he said. "You're basically having to interview tons of people just to find those who can pass it."

Parnin took issue with the way the industry has dealt with the difficulty of its tests. Rather than coming up with a fair way to evaluate software engineers, companies like Google advise at least 40 practice sessions - a time commitment that's not an option for everyone. This amounts to stress inoculation training and it does help people pass whiteboard tests, he said, but it doesn't make the tests an effective skill assessment tool.

As an alternative, the paper points to the way devops biz Honeycomb (Hound Technology) – overseen by a female CEO, CTO, CMO and VP of engineering – approaches hiring. The company provides interview questions in advance so it's not a Trier Social Stress Test.

As the company explains on its website, its goal is to avoid surprises. "The research is clear: unknowns cause anxiety, and people don’t perform well when they’re anxious," the company says.

"The big picture is to provide more accessible alternatives," said Parnin. "There are a lot of ways to test for the same thing without putting all this pressure on people." ®

Send us news
123 Comments

Terraform fork OpenTF renamed and relocated as OpenTofu

Open wide!

California governor vetoes bill requiring human drivers in robo trucks

Route 404: Human driver requirement not found

Automating cloud infrastructure: Do you want APIs with that?

Flipping the script to a control plane

GNU turns 40: Stallman's baby still not ready for prime time, but hey, there's cake

It turned the software industry upside down regardless

The Pentagon has no idea how to deal with bad cloud contracts, say auditors

Terrible IT practices at the DoD? You don't say

Rusty revenant Servo returns to render once more

Mozilla gave it the boot, but the Linux Foundation Europe gave it the kiss of life instead

Now IBM sued for age discrim by its own HR veterans

Staff with short 'runways' told to take off amid shift to corporate chatbots, it's claimed

Beneath Microsoft's Surface event, AI spreads everywhere

Windows gets its own Copilot to help operate the operating system – Edge, Bing, Outlook, 365 not spared, either

Devs learn rival Godot engine in a week to poke fun at Unity

The result? Install Fee Tycoon

Game on: FTC tries again to block Microsoft Activision merger

Redmond says the deal is still on

Long-term support for Linux kernels is about to get a lot shorter

Despite the OS's success, maintainers are short-staffed and under-appreciated

US Trademark Office still wants to keep faxes, but is willing to try this cloud thing

Finally, we've arrived in the future