Software

You're testing them wrong: Whiteboard coding interviews are 'anti-women psychological stress examinations'

Boffins find flaws in the way software engineering hopefuls are sifted


People applying for software engineering positions at companies are often asked to solve problems on a whiteboard, under the watchful eye of an interviewer, as a way to assess technical problem solving skills.

But recent research suggests that whiteboard technical tests – so daunting to job seekers that there are books on how to deal with them – often fail to assess technical skill, according to new research. Instead, they're all about pressure.

In a paper [PDF] to be presented later this year at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, researchers from North Carolina State University (NCSU) and Microsoft in the US argue that whiteboard sessions test for stage fright rather than, y'know, coding competency.

The title of the paper hints at its conclusion: "Does Stress Impact Technical Interview Performance?" NCSU authors Mahnaz Behroozi, Shivani Shirolkar, and Chris Parnin, with Titus Barik from Microsoft, say it most certainly does.

"Through a happy accident, the software industry has seemingly reinvented a crude yet effective instrument for reliably introducing stress in subjects, which typically manifests as performance anxiety," the paper explains.

When Facebook says you're not a good 'culture fit', it means you're not White or Asian enough – complaint

READ MORE

"A technical interview has an uncanny resemblance to the Trier Social Stress Test, a procedure used for decades by psychologists and is the best known 'gold standard' procedure for the sole purpose of reliably inducing stress."

As a consequence, whiteboard interviews may fail to assess coder competency. Rather, the researchers argue, they measure how well job candidates handle anxiety.

Using 48 graduate and undergraduate students with programming experience, the researchers conducted a randomized controlled trial to compare the traditional technical interview (done while being watched) with a private session evaluation (done without being observed). The experiment was designed to measure cognitive load and stress through the collection of eye tracking metrics, specifically fixation duration and pupil dilation.

The researchers found that stress hinders interview performance, with participants in the traditional technical interview exhibiting higher cognitive load, lower scores, and higher stress levels. In essence, social anxiety took otherwise qualified job candidates out of the running because of the circumstances of the interview.

Further flaws

What's more, whiteboard technical interviews appear to favor men over women.

"We also observed that no women successfully solved the problem in the public setting, whereas all women solved it correctly in the private setting," the paper says.

In a phone interview with The Register, Christopher Parnin, assistant professor at NC State University and one of the paper's co-authors, said he doesn't have a conclusive reason why this might be the case. He said there's some support in academic literature to indicate the women have more performance anxiety than men, but he stressed that's a gross oversimplification because men experience performance anxiety too.

For Parnin, the problem is whiteboard tests themselves. "It all comes down to the fact that the test is designed to make almost anyone fail," he said. "You're basically having to interview tons of people just to find those who can pass it."

Parnin took issue with the way the industry has dealt with the difficulty of its tests. Rather than coming up with a fair way to evaluate software engineers, companies like Google advise at least 40 practice sessions - a time commitment that's not an option for everyone. This amounts to stress inoculation training and it does help people pass whiteboard tests, he said, but it doesn't make the tests an effective skill assessment tool.

As an alternative, the paper points to the way devops biz Honeycomb (Hound Technology) – overseen by a female CEO, CTO, CMO and VP of engineering – approaches hiring. The company provides interview questions in advance so it's not a Trier Social Stress Test.

As the company explains on its website, its goal is to avoid surprises. "The research is clear: unknowns cause anxiety, and people don’t perform well when they’re anxious," the company says.

"The big picture is to provide more accessible alternatives," said Parnin. "There are a lot of ways to test for the same thing without putting all this pressure on people." ®

Send us news
123 Comments

Tesla Autopilot is a lot dumber than CEO Musk claims, says Cali DMV after speaking to the software's boss

'Elon's tweet does not match engineering reality' states poorly redacted report

Tesla CEO Elon Musk's public statements about the state of his automaker's Autopilot assistive driving technology overestimate the system's capabilities, according to documents released by the California Department of Motor Vehicles (DMV).

Legal non-profit PlainSite obtained the DMV documents via the California Public Records Act and they include a summary, written by Miguel Acosta, chief of the DMV's Autonomous Vehicles Branch, of a March 9, 2021 meeting between DMV officials and Tesla personnel.

Acosta wrote that "DMV asked CJ [CJ Moore, director of Autopilot software at Tesla] to address, from an engineering perspective, Elon’s messaging about L5 capability by the end of the year."

Continue reading

Facebook: Nice iOS app of ours you have there, would be a shame if you had to pay for it

Antisocial giant insists 'Help keep FB free of charge' messaging is merely educational

The number of Facebook and Instagram users on iOS agreeing to be tracked by the social networking behemoth for targeted ads has fallen drastically in the week since Apple's iOS 14.5 debuted – and Zuck & Co have hit back.

The App Tracking Transparency framework in iOS 14.5 requires companies to ask permission to observe the activities of iOS app users – that is to say, to link application usage and data with user or device information collected from other sources for targeted advertising or analytics.

This opt-in regime looks to be an extinction event for the current incarnation of targeted advertising, on iOS at least. According to analytics biz Flurry, only about 12 per cent of iOS users worldwide and only four per cent in the US have decided they want to be tracked.

Continue reading

Xpand your horizons: MariaDB launches distributed query engine into proprietary DBaaS

But beware lock-in-as-a-service, analyst warns

MariaDB has added proprietary bells and whistles, in the form of distributed SQL, for its DBaaS and supposedly developer-friendly front end.

The biz supporting the open-source MySQL-derived database introduced its DBaaS SkySQL last year and has now announced the general availability of its distributed SQL as one of the engines in MariaDB's SkySQL system, said CMO Franz Aman.

"What's cool about distributed SQL is that you get all the scale of NoSQL, but you get it with all the benefits of relational," he said. "So, you have strong consistency, you have full SQL vocabulary, but at a scale that is ready for the internet for internet-scale."

Continue reading

Russian cyber-spies changed tactics after the UK and US outed their techniques – so here's a list of those changes

Plus: NCSC warns of how hostile powers may exploit smart city infrastructure

Russian spies from APT29 responded to Western agencies outing their tactics by adopting a red-teaming tool to blend into targets' networks as a legitimate pentesting exercise.

Now, the UK's National Cyber Security Centre (NCSC) and the US warn, the SVR is busy exploiting a dozen critical-rated vulns (including RCEs) in equipment ranging from Cisco routers through to VMware virtualization kit – and the well-known Pulse Secure VPN flaw, among others.

"In one example identified by the NCSC, the actor had searched for authentication credentials in mailboxes, including passwords and PKI keys," warned the GCHQ offshoot today.

Continue reading

GitLab's 10-day certification freebie offer lasted only two because, surprise surprise, people really like freebies

Biz expected 4,000 signups, got 60,000, system couldn't cope

GitLab says a surge in demand and a technical shortcoming resulted in the DevOps outfit yanking a free certification offer barely two days after turning on the tap.

In a postmortem write-up this week, GitLab manager Christine Yoshida said the infrastructure of its glossy "learning experience ... eventually hit a system limit" as excited users piled on, and the promotion period was ended early.

A discount code was made available in April to people who wanted to get GitLab-certified. The 100 per cent discount was planned to last for ten days, and the GitLab gang figured 4,000 users would sign up.

Continue reading

British bank TSB says it will fix days-long transaction troubles tonight

Totally Sucks, Buddy: Debit payments held up since April, online and app still wobbly, say readers

TSB admitted today it still hadn't fixed a transaction processing issue that has for days held up customers' payments, with users continuing to have issues at the time of publication.

We're told the transaction hold-up, which the Edinburgh-based bank said was linked to debit accounts, would be resolved "overnight." It did claim to have fixed a "temperamental" technical fault preventing some customers from accessing their online accounts, however.

Reg readers who double up as customers of TSB – once known as the comedy bank because of the frequency at which its web-based services fell over – maintained they were still having troubles logging onto the app or website, with some having experienced issues for days.

Continue reading

Privacy activist Max Schrems on Microsoft's EU data move: It won't keep the NSA away

Software giant vows data processing of EU cloud services to stay in EU, which means that currently...

Microsoft has announced plans to ensure data processing of EU cloud services within the borders of the political bloc in a move that expert observers claim reveals problems with the firm's existing setup.

Those problems extend to UK public sector organisations seeking to stick within government guidance as well as a longstanding issue where personal data held in the EU can potentially be accessed via US security laws.

In a blog, Brad Smith, Microsoft’s president and chief legal officer, said the software and cloud services giant would, by the end 2022, enable EU customers of Azure, Microsoft 365, and Dynamics 365 to have all their data processed physically within the EU.

Continue reading

We were 'blindsided' by Epic's cheek, claims Apple exec on 4th day of antitrust wrangling

I thought we were friends

An Apple exec has spoken of his shock after Fortnite creator Epic Games installed a hotfix that allowed it to deploy its own payment methods, thus skirting the 30 per cent App Store tax.

Testifying on the fourth day of the bench trial, Apple's vice president of App Store, Matt Fischer, said he had been "blindsided" by the deployment of the workaround, given the amicable relationship previously enjoyed by both companies.

Fischer said (audio here) that Apple's marketing teams had previously promoted in-game events taking place within Fortnite involving DJ Marshmello and rapper Travis Scott. He also claimed that Cupertino had expressed a willingness to reconsider its prohibition on the in-game gifting of virtual items.

Continue reading

'A massive middle finger': Open-source audio fans up in arms after Audacity opts to add telemetry capture

Move comes days after firm acquired by Muse Group

Open source audio software outfit Audacity, now under new management, is adding some "basic telemetry", much to the alarm of many of its community.

The request turned up in GitHub this week, aimed at providing some telemetry, and the author of the request, Dmitry Vedenko, explained:

Continue reading

Broadband plumber Openreach yanks legacy copper phone lines in Suffolk town of Mildenhall en route to getting the UK on VoIP

Just four years to go before planned switch-off

The tiny Suffolk town of Mildenhall is the second place where Openreach has stopped selling copper products as the company develops its strategy for withdrawing legacy telephone lines.

The "stop-sell" order came into effect on 4 May, and also extends to copper-based phone connections. It follows a similar stop-sell edict in Salisbury, which last year became the first UK city to receive full-fibre coverage.

While this decision hasn't had an immediate impact on those hanging onto their slower copper lines, it has meant those hoping to switch providers or upgrade their connection will be pushed to a digital-only service.

Continue reading

Stale and past its best. Are you talking about Windows or the pizza you're waiting for?

A reminder of golden Start Menu days

Bork!Bork!Bork! There are certain things that do not belong in pizza. One is pineapple. Another is the Windows Start Menu.

Spotted by Reg reader Dean K, who was making an essential journey to collect the steaming disc of doughy delight from a well-known cheese-and-tomato bread slinger, the screen that would normally tell customers how long they had to wait was flashing something altogether more worrying.

In this instance, it was the Windows Start Menu from long ago. Long removed from current versions of the operating system (although Microsoft has continued to find ways of reintroducing it without admitting that the unmitigated disaster of the Windows 8 user interface was anything other than an adventure in touch-first design), the old thing is showing its wares for all to see.

Continue reading