Amazon continues its tsunami of announcements, now with AI in mind. We spoke to an AWS lead to decode it all

Inference can be expensive but the cloud giant's working on that, we're told

re:Invent Amazon Web Services is in the second week of its three-week re:Invent run, and the show's announcement deluge continues, now with an emphasis on AI-oriented services.

The Register on Wednesday spoke with Ian Massingham, worldwide lead for startup solutions architecture at AWS, in an attempt to make sense of it all.

On Tuesday, Swami Sivasubramanian, AWS VP of machine learning, presided over what the internet titan billed as its first-ever machine-learning-focused keynote – surely not its last of the sort. The webcast presentation expanded upon the AI-flavored offerings that trickled out in week one, the details of which can be found in AWS's exhaustive What's New list.

Massingham pointed to last week's introduction of Amazon EC2 G4ad Instances with AMD GPUs, noting that they can help drive down the cost of running machine learning models in the cloud.

"They're a really price-performant, efficient way for customers to run machine learning inference workloads," he said. "Using the integrated AMD GPU and CPU capacity should drive down the cost of ML inference workloads run on those systems."

AWS, a pioneer in cloud service bill shock, had that sort of thrift in mind when it acquired Israeli startup Annapurna Labs in 2015 to create its Inferentia chip to accelerate deep learning processing.

"It has enabled us to deliver really incredibly good price-performance on inference workloads for customers," Massingham said. "Bear in mind that the majority of cost associated with running machine learning workloads for customers comes in the inference component. You train the model once but you may make hundreds of thousands or millions of inference calls against that model every day if your service is successful and widely deployed."

Going all in

He also highlighted various industrial machine learning services introduced last week that provide companies with computer vision and machine monitoring geared toward detecting equipment anomalies, predictive maintenance, process control, and quality control. These include: Amazon Monitron, its Lookout for Equipment, AWS Panorama Appliance, and Amazon Lookout for Vision.

SageMaker, AWS's machine learning tool, saw a number of additions both this week and last. These include SageMaker Data Wrangler, a data preparation capability to identify inconsistencies, SageMaker Feature Store, a managed service for curating machine learning features for training and inference, and SageMaker Pipelines, which Massingham described as a CI/CD capability for automating workflows associated with training and deploying models and for running scheduled events.

More recently, SageMaker Clarify debuted to provide a way to detect bias in machine learning models and to make model-based predictions easier to explain. Massingham also called out Amazon SageMaker JumpStart which provides built-in solutions for common machine learning use cases and automated deployment of open source pre-trained ML models.

"The launches around SageMaker are likely to be the most impactful for the particular set of customers that I work with in the startup space," said Massingham.

"They really like the fact that SageMaker enables them to use ML capabilities to build and train models without having to build the toolchain themselves, or necessarily have a huge amount of machine learning expertise in house."

Home, or hotel maker

Asked to explain how AWS customers are actually using these tools, Massingham cited hotels.com as a long-time SageMaker user. One of the ways the company uses the service is to select which images its website shows to customers who are searching for a hotel room.

"If you're looking at property on hotels.com, they will select the images that they display in the property listing based on observations learned from other customers who have booked that property or similar properties," he said. "They have a long-term record of which images have been shown within each property listing that they have in their different channels, be that web or mobile."

The company's systems can learn from that dataset which types of images tend to lead to customer bookings. "So they'll place an image in the listing that's appropriate to drive a transaction from that customer," he explained.

Massingham also described how real estate industry customers use Sage Maker. "Realtor.com is using Sage maker to train models associated with real estate listings and with identifying characteristics and properties that might make [listings] particularly attractive to particular users or drive particularly high degrees of interest," he said.

AWS has also given some thought to the shortcomings of machine learning. Massingham pointed to the recent debut of Contact Lens for Amazon Connect, a portfolio of services aimed at call centers.

The new service, he explained, allows call center firms to detect when managerial intervention might be necessary during a customer contact session.

"Say you've got a conversation taking place within a contact center," he said. "You can use this service to detect when a customer might be upset or angry or agitated in a conversation with a [human] agent. And you can interject a manager into that conversation automatically, on the basis of that detection event."

So next time you need to speak with a manager, shouting and mouthing off really will work, if there's an Amazon AI listening. ®