Google isn't even trying to not be creepy: 'Continuous Match Mode' in Assistant will listen to everything until it's disabled

I always feel like / somebody's dropping eaves / and I have no privacy


Google has introduced "continuous match mode" for apps on its voice-powered Assistant platform, where it will listen to everything without pausing. At the same time it has debuted related developer tools, new features, and the ability to display web content on its Smart Display hardware using the AMP component framework.

The Chocolate Factory has big plans for its voice assistant. "We consider voice to be the biggest paradigm shift around us," said director of product Baris Gultekin, speaking at the Voice Global summit, where the new features were introduced.

The goal is "ambient computing", where you can interact with the big G anywhere at any time, so pervasively that you do not notice it. Voice interaction is a key part of this since it extends the ability to perform searches or run applications to scenarios where tapping a keyboard or touching a display are not possible.

Google Assistant exists in many guises such as on smartphones and watches, TVs, PCs, and also on dedicated hardware, such as the voice-only Google Home and Google Home Mini, or with "smart display" screens on the Google Nest Hub or devices from Lenovo and Harman. While assistant devices have been popular, Android phones (which nag you to set up the Assistant) must form the largest subset of users. Over all the device types, the company claims over 500 million active users.

As with any evolving platform, developer support is critical. At the Voice Global event, the company introduced new tooling, the biggest piece being the Actions Builder, a web-based tool that lets you add and assemble the pieces of an "action", essentially an application for Assistant. Actions Builder is an easy-to-use tool that integrates with Natural Language Understanding for parsing user input into what makes sense for an action, and lets you create the flow of intents and responses that form a complete action.

Payam Shodjai, senior director of product for Google Assistant, explained at the summit that Builder is a response to complaints that the previous approach, where developers switched back and forth between an Actions Console and a DialogFlow editor, was too disjointed.

Actions Builder will "replace DialogFlow as the preferred way to develop actions on the assistant," said Shodjai.

Google's new Action Builder at work

Google's new Action Builder at work

Trying out the new Action Builder, we discovered that running an action under development is impossible if you have the Web and App Activity permission, which lets Google keep a record of your actions, disabled. A dialog appears prompting you to enable it. It is a reminder of how Google Assistant is entwined with the notion that you give Google your data in return for personalised experiences.

Some developers prefer to work with text-based code rather than a visual builder, and Google is responding to this need with an updated Actions SDK which lets you work in your preferred IDE or programmer's editor. This also makes it easier to use source control and continuous integration tools with action projects.

There are also new features in the Assistant platform. Home Storage is a way of storing values across multiple sessions in a single household, as defined by Google's Home Graph, part of its smart home API. This is significant because it enables multi-user applications. The examples demonstrated were games such as word puzzles.

DOn't say we didn't warn you

The documents reveal a strong warning, though, in red type: "Important: Do not use home storage to store any personal data. Home storage data applies across multiple individuals in the same household who use the same action."

It will be down to developers to heed such warnings.

The Media API for Google Assistant has been updated and now handles play and resume across devices, and the general ability to start playback from a specific point rather than from the beginning.

Devices with Smart Displays are gaining the ability to display web content – provided it is delivered using the AMP component framework. Initial support will be for news sites, coming later this summer, but with more categories to follow.

The most intriguing new feature is called Continuous Match Mode. This touches on an area of debate, which is the extent an Assistant device listens in on conversations without specifically being asked to do so. Currently you have to use a wake word to initialise an Assistant action, though devices must be listening at all times to some extent in order to recognise the wake word – "Hey Google" or "OK Google" – which can be followed by a custom invocation for a third-party action.

Once in a conversation – the equivalent to running an application – the system follows a flow based on parsing the user's words into intents that are meaningful in the the context of the conversation. The action continues to completion of the flow, or quits if the action cannot parse the user's intents, or after a system command such as "cancel". The developer can override the first exit command with one last plea to continue, but not the second.

"Sometimes you want to build experiences that enable the mic to remain open, to enable users to speak more naturally with your action, without waiting for a change in mic states," said Shodjai at the summit and in the developer post.

"Today we are announcing an early access program for Continuous Match Mode, which allows the assistant to respond immediately to user's speech enabling more natural and fluid experiences. This is done transparently, so that before the mic opens the assistant will announce, 'the mic will stay open temporarily', so users know they can now speak freely without waiting for additional prompts."

The mode is not yet publicly documented. The demonstrated example was for a game with jolly cartoon pictures; but there may be privacy implications since in effect this setting lets the action continue to listen to everything while the mode is active.

Shodjai did not explain how users will end a Continuous Match Mode session but presumably this will be either after a developer-defined exit intent, or via a system intent as with existing actions. Until that happens, the action will be able to keep running.

Just as with personalisation via tracking and data collection, privacy and pervasive computing do not sit comfortably together, and with the new Continuous Match Mode a little more privacy slips away. ®

Similar topics

Broader topics


Other stories you might like

  • Google has more reasons why it doesn't like antitrust law that affects Google
    It'll ruin Gmail, claims web ads giant

    Google has a fresh list of reasons why it opposes tech antitrust legislation making its way through Congress but, like others who've expressed discontent, the ad giant's complaints leave out mention of portions of the proposed law that address said gripes.

    The law bill in question is S.2992, the Senate version of the American Innovation and Choice Online Act (AICOA), which is closer than ever to getting votes in the House and Senate, which could see it advanced to President Biden's desk.

    AICOA prohibits tech companies above a certain size from favoring their own products and services over their competitors. It applies to businesses considered "critical trading partners," meaning the company controls access to a platform through which business users reach their customers. Google, Apple, Amazon, and Meta in one way or another seemingly fall under the scope of this US legislation. 

    Continue reading
  • I was fired for blowing the whistle on cult's status in Google unit, says contractor
    The internet giant, a doomsday religious sect, and a lawsuit in Silicon Valley

    A former Google video producer has sued the internet giant alleging he was unfairly fired for blowing the whistle on a religious sect that had all but taken over his business unit. 

    The lawsuit demands a jury trial and financial restitution for "religious discrimination, wrongful termination, retaliation and related causes of action." It alleges Peter Lubbers, director of the Google Developer Studio (GDS) film group in which 34-year-old plaintiff Kevin Lloyd worked, is not only a member of The Fellowship of Friends, the exec was influential in growing the studio into a team that, in essence, funneled money back to the fellowship.

    In his complaint [PDF], filed in a California Superior Court in Silicon Valley, Lloyd lays down a case that he was fired for expressing concerns over the fellowship's influence at Google, specifically in the GDS. When these concerns were reported to a manager, Lloyd was told to drop the issue or risk losing his job, it is claimed. 

    Continue reading
  • Makers of ad blockers and browser privacy extensions fear the end is near
    Overhaul of Chrome add-ons set for January, Google says it's for all our own good

    Special report Seven months from now, assuming all goes as planned, Google Chrome will drop support for its legacy extension platform, known as Manifest v2 (Mv2). This is significant if you use a browser extension to, for instance, filter out certain kinds of content and safeguard your privacy.

    Google's Chrome Web Store is supposed to stop accepting Mv2 extension submissions sometime this month. As of January 2023, Chrome will stop running extensions created using Mv2, with limited exceptions for enterprise versions of Chrome operating under corporate policy. And by June 2023, even enterprise versions of Chrome will prevent Mv2 extensions from running.

    The anticipated result will be fewer extensions and less innovation, according to several extension developers.

    Continue reading
  • End of the road for biz living off free G Suite legacy edition
    Firms accustomed to freebies miffed that web giant's largess doesn't last

    After offering free G Suite apps for more than a decade, Google next week plans to discontinue its legacy service – which hasn't been offered to new customers since 2012 – and force business users to transition to a paid subscription for the service's successor, Google Workspace.

    "For businesses, the G Suite legacy free edition will no longer be available after June 27, 2022," Google explains in its support document. "Your account will be automatically transitioned to a paid Google Workspace subscription where we continue to deliver new capabilities to help businesses transform the way they work."

    Small business owners who have relied on the G Suite legacy free edition aren't thrilled that they will have to pay for Workspace or migrate to a rival like Microsoft, which happens to be actively encouraging defectors. As noted by The New York Times on Monday, the approaching deadline has elicited complaints from small firms that bet on Google's cloud productivity apps in the 2006-2012 period and have enjoyed the lack of billing since then.

    Continue reading
  • It's a crime to use Google Analytics, watchdog tells Italian website
    Because data flows into the United States, not because of that user interface

    Another kicking has been leveled at American tech giants by EU regulators as Italy's data protection authority ruled against transfers of data to the US using Google Analytics.

    The ruling by the Garante was made yesterday as regulators took a close look at a website operator who was using Google Analytics. The regulators found that the site collected all manner of information.

    So far, so normal. Google Analytics is commonly used by websites to analyze traffic. Others exist, but Google's is very much the big beast. It also performs its analysis in the USA, which is what EU regulators have taken exception to. The place is, after all, "a country without an adequate level of data protection," according to the regulator.

    Continue reading
  • UK competition watchdog seeks to make mobile browsers, cloud gaming and payments more competitive
    Investigation could help end WebKit monoculture on iOS devices

    The United Kingdom's Competition and Markets Authority (CMA) on Friday said it intends to launch an investigation of Apple's and Google's market power with respect to mobile browsers and cloud gaming, and to take enforcement action against Google for its app store payment practices.

    "When it comes to how people use mobile phones, Apple and Google hold all the cards," said Andrea Coscelli, Chief Executive of the CMA, in a statement. "As good as many of their services and products are, their strong grip on mobile ecosystems allows them to shut out competitors, holding back the British tech sector and limiting choice."

    The decision to open a formal investigation follows the CMA's year-long study of the mobile ecosystem. The competition watchdog's findings have been published in a report that concludes Apple and Google have a duopoly that limits competition.

    Continue reading
  • Google recasts Anthos with hitch to AWS Outposts
    If at first you don't succeed, change names and try again

    Google Cloud's Anthos on-prem platform is getting a new home under the search giant’s recently announced Google Distributed Cloud (GDC) portfolio, where it will live on as a software-based competitor to AWS Outposts and Microsoft Azure Stack.

    Introduced last fall, GDC enables customers to deploy managed servers and software in private datacenters and at communication service provider or on the edge.

    Its latest update sees Google reposition Anthos on-prem, introduced back in 2020, as the bring-your-own-server edition of GDC. Using the service, customers can extend Google Cloud-style management and services to applications running on-prem.

    Continue reading
  • Google offers $118m to settle gender discrimination lawsuit
    Don't even think about putting LaMDA on the compensation committee

    Google has promised to cough up $118 million to settle a years-long gender-discrimination class-action lawsuit that alleged the internet giant unfairly pays men more than women.

    The case, launched in 2017, was led by three women, Kelly Ellis, Holly Pease, and Kelli Wisuri, who filed a complaint alleging the search giant hires women in lower-paying positions compared to men despite them having the same qualifications. Female staff are also less likely to get promoted, it was claimed.

    Gender discrimination also exists within the same job tier, too, the complaint stated. Google was accused of paying women less than their male counterparts despite them doing the same work. The lawsuit was later upgraded to a class-action status when a fourth woman, Heidi Lamar, joined as a plaintiff. The class is said to cover more than 15,000 people.

    Continue reading
  • Google: How we tackled this iPhone, Android spyware
    Watching people's every move and collecting their info – not on our watch, says web ads giant

    Spyware developed by Italian firm RCS Labs was used to target cellphones in Italy and Kazakhstan — in some cases with an assist from the victims' cellular network providers, according to Google's Threat Analysis Group (TAG).

    RCS Labs customers include law-enforcement agencies worldwide, according to the vendor's website. It's one of more than 30 outfits Google researchers are tracking that sell exploits or surveillance capabilities to government-backed groups. And we're told this particular spyware runs on both iOS and Android phones.

    We understand this particular campaign of espionage involving RCS's spyware was documented last week by Lookout, which dubbed the toolkit "Hermit." We're told it is potentially capable of spying on the victims' chat apps, camera and microphone, contacts book and calendars, browser, and clipboard, and beam that info back to base. It's said that Italian authorities have used this tool in tackling corruption cases, and the Kazakh government has had its hands on it, too.

    Continue reading
  • Brave Search leaves beta, offers Goggles for filtering, personalizing results
    Freedom or echo chamber?

    Brave Software, maker of a privacy-oriented browser, on Wednesday said its surging search service has exited beta testing while its Goggles search personalization system has entered beta testing.

    Brave Search, which debuted a year ago, has received 2.5 billion search queries since then, apparently, and based on current monthly totals is expected to handle twice as many over the next year. The search service is available in the Brave browser and in other browsers by visiting search.brave.com.

    "Since launching one year ago, Brave Search has prioritized independence and innovation in order to give users the privacy they deserve," wrote Josep Pujol, chief of search at Brave. "The web is changing, and our incredible growth shows that there is demand for a new player that puts users first."

    Continue reading

Biting the hand that feeds IT © 1998–2022