AI-on-demand as Google Cloud TPUs are rentable for a few bucks a hour

Service is now available across three continents for machine-learning code eggheads


AI developers can now rent Google’s Cloud TPU chips in the US, Asia, and Europe by the hour.

The Cloud TPU, also referred to as TPU2, is the second generation of Google’s TPU hardware aimed at optimizing machine learning models written in TensorFlow. The first TPU was more of an internal experiment and could only handle inference. The second TPU is made for training and inference, and is the first chip that is available for cloud customers.

Now, from this week, developers in the United States, Asia, and Europe can rent them out by the hour. The price can drop by 70 per cent for those willing to sign up for Google’s preemptible service, where users run the risk of being kicked off the service if demand rises.

TPU_price

Cloud TPU2 pricing across three different regions for their normal and preemptible service ... click to enlarge

It works in a similar way to Amazon Web Service’s GPU reserved instances (RI) and its on-demand (OD) spot pricing service. The P3 instances uses Nvidia’s latest Tesla V100 chips.

aws_gpu_pricing

AWS P3 pricing for Tesla V100 chips for its reserved and on demand services ... Source: The Next Platform

There are advantages to using AWS since its available in more regions, including: US East (N. Virginia), US West (Oregon), US East (Ohio), Europe West (Ireland), Asia Pacific (Tokyo), Asia Pacific (Beijing), Asia Pacific (Seoul) and GovCloud (US) regions. It’s also more flexible and works for systems that are written in other frameworks other than TensorFlow.

Also, don't forget Microsoft Azure offers GPU math accelerators in the cloud. And don't forget, cloud giants can run out of GPU resources.

Comparing chips can be difficult as companies don’t really like to disclose explicit benchmarks. Projects like DAWNBench, a competition that encourages developers to submit models optimizing for training time, and costs to carry out training and inference, and inference latency are interesting and useful.

Google engineers entered and won some of the entries for cheapest and fastest ImageNet training with a ResNet-50 model. It’s AmoebaNet model also featured in the leaderboards for image classification.

Here’s how much it costs to train some of Google’s open-source models:

cloud_TPU_models

Cost to train some popular open-source models on using Cloud TPUs using the normal and preemptible service.

It would be interesting to see how this compared with other cloud services, but it’s a little tricky since the same models can be optimized across different frameworks. Startups like RiseML have tried to have a crack at it, and it turns out the difference in speed between the Tesla V100 and Cloud TPUs is pretty small.

There is also the MLPerf project that measures the speed of training and inference times for popular machine learning models across a variety of chips and frameworks to look forward to. Engineers from Google, Intel, Baidu, AMD, other hardware startups like SambaNova and Wave Computing, and universities are currently submitting their models in the competition until the July 31 deadline. ®

Similar topics


Other stories you might like

  • Battlefield 2042: Please don't be the death knell of the franchise, please don't be the death knell of the franchise

    Another terrible launch, but DICE is already working on improvements

    The RPG Greetings, traveller, and welcome back to The Register Plays Games, our monthly gaming column. Since the last edition on New World, we hit level cap and the "endgame". Around this time, item duping exploits became rife and every attempt Amazon Games made to fix it just broke something else. The post-level 60 "watermark" system for gear drops is also infuriating and tedious, but not something we were able to address in the column. So bear these things in mind if you were ever tempted. On that note, it's time to look at another newly released shit show – Battlefield 2042.

    I wanted to love Battlefield 2042, I really did. After the bum note of the first-person shooter (FPS) franchise's return to Second World War theatres with Battlefield V (2018), I stupidly assumed the next entry from EA-owned Swedish developer DICE would be a return to form. I was wrong.

    The multiplayer military FPS market is dominated by two forces: Activision's Call of Duty (COD) series and EA's Battlefield. Fans of each franchise are loyal to the point of zealotry with little crossover between player bases. Here's where I stand: COD jumped the shark with Modern Warfare 2 in 2009. It's flip-flopped from WW2 to present-day combat and back again, tried sci-fi, and even the Battle Royale trend with the free-to-play Call of Duty: Warzone (2020), which has been thoroughly ruined by hackers and developer inaction.

    Continue reading
  • American diplomats' iPhones reportedly compromised by NSO Group intrusion software

    Reuters claims nine State Department employees outside the US had their devices hacked

    The Apple iPhones of at least nine US State Department officials were compromised by an unidentified entity using NSO Group's Pegasus spyware, according to a report published Friday by Reuters.

    NSO Group in an email to The Register said it has blocked an unnamed customers' access to its system upon receiving an inquiry about the incident but has yet to confirm whether its software was involved.

    "Once the inquiry was received, and before any investigation under our compliance policy, we have decided to immediately terminate relevant customers’ access to the system, due to the severity of the allegations," an NSO spokesperson told The Register in an email. "To this point, we haven’t received any information nor the phone numbers, nor any indication that NSO’s tools were used in this case."

    Continue reading
  • Utility biz Delta-Montrose Electric Association loses billing capability and two decades of records after cyber attack

    All together now - R, A, N, S, O...

    A US utility company based in Colorado was hit by a ransomware attack in November that wiped out two decades' worth of records and knocked out billing systems that won't be restored until next week at the earliest.

    The attack was detailed by the Delta-Montrose Electric Association (DMEA) in a post on its website explaining that current customers won't be penalised for being unable to pay their bills because of the incident.

    "We are a victim of a malicious cyber security attack. In the middle of an investigation, that is as far as I’m willing to go," DMEA chief exec Alyssa Clemsen Roberts told a public board meeting, as reported by a local paper.

    Continue reading

Biting the hand that feeds IT © 1998–2021