Zuckerberg's absolutely mental: Brain sensors that read YOUR MIND at 100 words a minute

Billionaire dreamer to see your thoughts 'by 2019'


F8 2017 Facebook sees such promise in virtual reality that it has taken to celebrating products and services that don't exist outside laboratory settings, like brain interfaces, augmented reality glasses, and "hearing" through one's skin.

During the second day of its F8 developer conference on Wednesday, representatives of the social media giant provided an update on the company's ten-year technology roadmap. Last year, CEO Mark Zuckerberg summarized his plan thus: "Give everyone the power to share anything with anyone."

The Facebook-broadcast killing of a Cleveland man over the weekend suggests the need to modify that aspiration with a few caveats.

In years past, tech companies pitching vaporware drew scorn. But in the selfie era Facebook, like Google, can't resist sharing schemes to save the world.

Such behavior is understandable as a way to make the company more appealing to the public, to potential employees, and to investors. But it comes without commitment to any specific deliverables or the need to allay concerns about dystopian fallout.

Regina Dugan, who leads Facebook's Building 8 hardware research group, closed Facebook's forward-looking keynote with a glimpse of the company's effort to build "a brain mouse for AR."

The goal, she said, was to create a system that "can type 100 words per minute straight from your brain." Facebook aims to reach this milestone in two years.

The current state of the art is about eight words per minute, based on medically oriented work overseen by Krishna Shenoy at Stanford's Neural Prosthetic Systems Lab. However, Dugan conceded that these systems don't yet operate in real time and they require surgery. "That simply won't scale," she said, evidently unsold on crowdsourcing the scalpel work.

Dugan said the technology to read thoughts doesn't exist today, but she expects advances in optical imaging will help make non-invasive brain interfaces viable.

"What if we make it possible to hear through your skin?" Dugan then asked, describing a system by which spoken words can be translated into localized vibration patterns. A Facebook employee identified as Francis has managed to learn nine words in this manner.

"She has learned to feel the acoustic shape on her arm," said Dugan. "She processes these shapes in her brain as words. She's learning how to use the artificial cochlea [a haptic sleeve] we made for her skin."

Dugan suggested that such technology could one day allow someone to think in Mandarin and convey meaning to a conversation partner in Spanish – the semantic meaning behind words would be transmitted hapticly, independent of language.

An even more compelling application would be cheating at cards. A spotter could secretly convey information to a colleague without a visible earpiece or a covert video feed in eyeglasses. If Las Vegas casinos haven't yet invested in thermal cameras capable of detecting electronics embedded in clothing or skin, they'd be well-advised to consider the possibilities.

But again, as Dugan observed, "These things are still a few years away."

Those who recall the word "privacy" may be heartened to learn that Facebook isn't presently interested in reading all your thoughts – despite the obvious value to marketers. It's only concerned with thoughts you want to share.

"To be clear, we are not talking about decoding your random thoughts," she said. "That might be more than any of us care to know. And it's not something any of us should have a right to know."

Leave that to Palantir.

Michael Abrash, chief scientist for Facebook's Oculus, extolled the potential of augmented reality, even as he poured cold water on the prospect of seeing AR glasses anytime soon.

"The true breakthrough will come when the real and virtual worlds can mix freely, wherever we are, whatever we're doing, so that the virtual world simply becomes part of our everyday reality," said Abrash. "That will require AR glasses and those will be much more technologically challenging than VR headsets. In fact, the set of technologies needed to build them doesn't yet exist."

Those who have invested in AR-darling Magic Leap may not want to hear that.

Abrash predicted it will be another 20 or 30 years before we have true AR capabilities in glasses. In the meantime, Facebook's Camera Effects Platform, introduced at F8 on Tuesday, has been created to dress up boring images of daily life with AR graphics, making them interesting enough to share on Facebook and ensuring a sufficient crop of attentive eyeballs for ad selling.

Facebook F8 x6 and x24 cameras

Joaquin Quiñonero Candela, director of applied machine learning at Facebook, highlighted Facebook's advances using machine learning to understand the vast amount of video streaming through, and stored on, Facebook's servers.

He said the company has already made great strides in terms of recognizing objects in images and being able to separate them from backgrounds – something a few years ago would have required meticulous manual labor in Adobe Photoshop. Its decision to open source its Caffe2 machine learning framework should bring further improvements.

While Facebook's systems aren't yet smart enough to block a live-streamed murder, that's an obviously desirable goal, both for Facebook and for censorious regimes around the world.

Yael Maguire, engineering director and head of Facebook's Connectivity Lab, announced three new wireless data transfer records: 36Gbps over 13km using millimeter wave technology, 80Gbps over the same distance with the company's optical cross-link technology, and 16Gbps bidirectionally from a Cessna flying over 7km (4.3mi) away.

Maguire also introduced a Tether-tenna, a small helicopter tethered to the ground with a wire that carries power and wireless data hardware. It's designed to provide instant network infrastructure in emergency situations.

Facebook CTO Michael Schroepfer, who opened the preview of things to come, showed off two 360-degree camera designs, the x24 and x6, for capturing VR imagery. The cameras are designed to work with Facebook's new 360 Capture SDK, designed to help developers integrate 360-degree imagery into their apps.

Facebook's hardware and software can in part solve the problem posed by fixed 360-degree cameras in VR applications – the fact that images captured from a specific point in space don't necessarily contain all the information necessary to view the scene from every angle. Its system manages this feat by filling in the blank areas not captured from a given viewpoint, to the extent that it can.

"Because of the quality of depth information, the resolution, and the arrangement of the physical hardware of the camera, and the quality of the computer vision code on the backend, we're able to create views that didn't exist before, to give you that sense of immersion [when you move around]," said Schroepfer.

If and when these technologies mature, Facebook may turn some of them into sources of revenue that didn't exist before. But why talk about something as grubby as ad selling when you can muse about connecting people? ®

Narrower topics


Other stories you might like

  • New York City rips out last city-owned public payphones
    Y'know, those large cellphones fixed in place that you share with everyone and have to put coins in. Y'know, those metal disks representing...

    New York City this week ripped out its last municipally-owned payphones from Times Square to make room for Wi-Fi kiosks from city infrastructure project LinkNYC.

    "NYC's last free-standing payphones were removed today; they'll be replaced with a Link, boosting accessibility and connectivity across the city," LinkNYC said via Twitter.

    Manhattan Borough President Mark Levine said, "Truly the end of an era but also, hopefully, the start of a new one with more equity in technology access!"

    Continue reading
  • Cheers ransomware hits VMware ESXi systems
    Now we can say extortionware has jumped the shark

    Another ransomware strain is targeting VMware ESXi servers, which have been the focus of extortionists and other miscreants in recent months.

    ESXi, a bare-metal hypervisor used by a broad range of organizations throughout the world, has become the target of such ransomware families as LockBit, Hive, and RansomEXX. The ubiquitous use of the technology, and the size of some companies that use it has made it an efficient way for crooks to infect large numbers of virtualized systems and connected devices and equipment, according to researchers with Trend Micro.

    "ESXi is widely used in enterprise settings for server virtualization," Trend Micro noted in a write-up this week. "It is therefore a popular target for ransomware attacks … Compromising ESXi servers has been a scheme used by some notorious cybercriminal groups because it is a means to swiftly spread the ransomware to many devices."

    Continue reading
  • Twitter founder Dorsey beats hasty retweet from the board
    We'll see you around the Block

    Twitter has officially entered the post-Dorsey age: its founder and two-time CEO's board term expired Wednesday, marking the first time the social media company hasn't had him around in some capacity.

    Jack Dorsey announced his resignation as Twitter chief exec in November 2021, and passed the baton to Parag Agrawal while remaining on the board. Now that board term has ended, and Dorsey has stepped down as expected. Agrawal has taken Dorsey's board seat; Salesforce co-CEO Bret Taylor has assumed the role of Twitter's board chair. 

    In his resignation announcement, Dorsey – who co-founded and is CEO of Block (formerly Square) – said having founders leading the companies they created can be severely limiting for an organization and can serve as a single point of failure. "I believe it's critical a company can stand on its own, free of its founder's influence or direction," Dorsey said. He didn't respond to a request for further comment today. 

    Continue reading
  • Snowflake stock drops as some top customers cut usage
    You might say its valuation is melting away

    IPO darling Snowflake's share price took a beating in an already bearish market for tech stocks after filing weaker than expected financial guidance amid a slowdown in orders from some of its largest customers.

    For its first quarter of fiscal 2023, ended April 30, Snowflake's revenue grew 85 percent year-on-year to $422.4 million. The company made an operating loss of $188.8 million, albeit down from $205.6 million a year ago.

    Although surpassing revenue expectations, the cloud-based data warehousing business saw its valuation tumble 16 percent in extended trading on Wednesday. Its stock price dived from $133 apiece to $117 in after-hours trading, and today is cruising back at $127. That stumble arrived amid a general tech stock sell-off some observers said was overdue.

    Continue reading
  • Amazon investors nuke proposed ethics overhaul and say yes to $212m CEO pay
    Workplace safety, labor organizing, sustainability and, um, wage 'fairness' all struck down in vote

    Amazon CEO Andy Jassy's first shareholder meeting was a rousing success for Amazon leadership and Jassy's bank account. But for activist investors intent on making Amazon more open and transparent, it was nothing short of a disaster.

    While actual voting results haven't been released yet, Amazon general counsel David Zapolsky told Reuters that stock owners voted down fifteen shareholder resolutions addressing topics including workplace safety, labor organizing, sustainability, and pay fairness. Amazon's board recommended voting no on all of the proposals.

    Jassy and the board scored additional victories in the form of shareholder approval for board appointments, executive compensation and a 20-for-1 stock split. Jassy's executive compensation package, which is tied to Amazon stock price and mostly delivered as stock awards over a multi-year period, was $212 million in 2021. 

    Continue reading
  • Confirmed: Broadcom, VMware agree to $61b merger
    Unless anyone out there can make a better offer. Oh, Elon?

    Broadcom has confirmed it intends to acquire VMware in a deal that looks set to be worth $61 billion, if it goes ahead: the agreement provides for a “go-shop” provision under which the virtualization giant may solicit alternative offers.

    Rumors of the proposed merger emerged earlier this week, amid much speculation, but neither of the companies was prepared to comment on the deal before today, when it was disclosed that the boards of directors of both organizations have unanimously approved the agreement.

    Michael Dell and Silver Lake investors, which own just over half of the outstanding shares in VMware between both, have apparently signed support agreements to vote in favor of the transaction, so long as the VMware board continues to recommend the proposed transaction with chip designer Broadcom.

    Continue reading

Biting the hand that feeds IT © 1998–2022