AWS debuts generative AI stuff so smart you might give it your money twice

Bedrock offers APIs for multiple models, plus EC2 instances for home-grown LLMs

Updated On Thursday Amazon Web Services announced an API platform named Bedrock, which hosts generative AI models built by top startups AI21 Labs, Anthropic, and Stability AI.

Generative AI has exploded in popularity with the development of models capable of producing text and images. Commercial tools developed by buzzy startups like OpenAI and Midjourney have won tens of millions of users, and Big Tech is now rushing to catch up. 

While Microsoft and Google compete to bring generative AI chatbots to search and productivity suites, Amazon's strategy is to remain fairly neutral – like some kind of machine-learning Switzerland – and provide access to the latest models on its cloud platform. It's a win-win for startups that have agreed to work with the e-commerce giant. Developers pay to use APIs to access the upstarts' models, and AWS provides all the underlying infrastructure that fully manages and provides those services.

"Customers have told us there are a few big things standing in their way today," said Swami Sivasubramanian, AWS' veep of machine learning, in a blog post.

"First, they need a straightforward way to find and access high-performing [foundational models] that give outstanding results and are best-suited for their purposes. Second, customers want integration into applications to be seamless, without having to manage huge clusters of infrastructure or incur large costs."

Amazon Bedrock currently offers large language models capable of processing and generating text – AI21 Labs' Jurassic-2 and Anthropic's Claude – and Stability AI's text-to-image model Stable Diffusion. Bedrock will also provide two of Amazon's own foundation models under the Titan brand, not to be confused with Google's Titan-branded stuff.

Developers can build their own generative AI-powered products and services on the backs of these Bedrock-managed APIs and can fine-tune a model for a particular task by providing their own labelled training examples. Amazon said this customization process will allow orgs to tailor neutral networks to their particular applications without having to worry if their private training data will leak, be misplaced, or be used to train other large language models.

Amazon is also pushing its family of custom-designed machine-learning acceleration chips Trainium and Inferentia, which can be used to train and run these kinds of generative models yourself in its cloud. A new EC2 instance type named Trn1 is powered by Trainium silicon, and developers can reportedly use it to save up to 50 percent on training costs compared to other EC2 instances.

Trn1 instances are optimized to distribute training across multiple servers and have a network bandwidth of up to 1600 Gbps. Developers can also spin up "ultraclusters" scaling up to 30,000 Trainium chips to deliver more than 6 exaflops of compute.

And for inference work, AWS says its Inf2 EC2 instance type is now generally available and accelerated by the cloud giant's Inferentia2 silicon.

Finally, Amazon is also making its AI pair-programming tool CodeWhisperer generally available and free to use for qualifying developers. It has been expanded to support ten new languages, including Go, Kotlin, Rust, PHP, SQL and more on top of the Python, Java, JavaScript, TypeScript, and C#.

"We believe CodeWhisperer is now the most accurate, fastest, and most secure way to generate code for AWS services, including Amazon EC2, AWS Lambda, and Amazon S3," Sivasubramanian opined. ®

Update

We noticed in Amazon's announcement it said "customers simply point Bedrock at a few labeled examples in Amazon S3, and the service can fine-tune the model for a particular task." That made us wonder if Bedrock has to be used with other AWS resources, both of which customers would have to pay for.

Representatives for AWS have been in touch to say Bedrock is a standalone fully managed API service that does not need any other AWS resources, such as S3, though customers can use Bedrock APIs and AWS services together if they wish, and will be billed by Amazon as necessary. Or customers can use just Bedrock on its own and pay for just that.