Building your own PC for AI is 10x cheaper than renting out GPUs on cloud, apparently

Here's the recipe for cooking up your own AMD-Nvidia beast


So, you’ve hunkered down and finally completed that online course on machine learning. It took weeks. Now, you have all sorts of ideas running through your mind on developing your own intelligent code and neural networks.

You assume you'll have to fork out a considerable wedge for a decent GPU-powered number-crunching rig, because your handy lightweight laptop is not going to cut it during the intensive network training process. So, seeing as you'll dabble with this on and off initially, you're looking at renting out GPUs on cloud.

Your heart drops a little when you total up the cloud instance costs. AI is cool and all, but is it worth forking out hundreds or even thousands of dollars for running your code on a remote platform? Probably not. There is a cheaper alternative: build your own machine-learning computer. This is, apparently, ten times cheaper than relying on cloud platforms like Amazon Web Services (AWS), according to one engineer.

Jeff Chen, an AI techie and entrepreneur at Stanford University in the US, believes that a suitable machine can be built for about $3,000 (~£2,300) without including tax. At the heart of the beast is an Nvidia GeForce 1080Ti GPU, a 12-core AMD Threadripper processor, 64GB of RAM, and a 1TB SSD card for data. Bung in a fan to keep the computer cool, a motherboard, a power supply, wrap the whole thing in a case, and voila.

Here’s the full checklist...

GPU_computer_cost

Image credit: Jeff Chen

Unlike renting out compute and data storage on cloud, once your personal rig is built, the only recurring cost to pay for is power. It costs $3 (£2.28) an hour to rent a GPU-accelerated system on AWS, whereas it’s only 20 cents (15p) to run on your own computer. Chen has done the sums, and, apparently, after two months that will work out to being ten times cheaper. The gap decreases slightly over time as the computer hardware depreciates.

“There are some drawbacks, such as slower download speed to your machine because it’s not on the backbone, static IP is required to access it away from your house, you may want to refresh the GPUs in a couple of years, but the cost savings is so ridiculous it’s still worth it,” he said this week.

An Nvidia GeForce 1080Ti isn’t the best GPU on the market, but it’ll be fine for most people unless you’re dealing with extremely large datasets and models. In fact, the performance on your computer is probably quite similar to the beefier GPU models like Nvidia’s Tesla V100, on cloud. A unit will set you back from $8,000 to over $10,000, compared to the measly $700 for a GeForce 1080Ti.

“The datacenter grade V100 is 25 to 75 per cent faster depending on workload, you don’t get the 4 to 8x [speedup] promised because you get bottlenecked by Amdahls law and slow IO. You get more memory but 11Gb on the 1080 Ti should be good for most cases,” Chen told The Register.

Nvidia has actually banned the use of cheap GeForce chips for datacenters since it updated its end-user licence agreement last year, forcing cloud providers to splash out on its more expensive options.

Jensen Huang, CEO of Nvidia, said GeForce cards were only really suited for gaming and cryptomining. But that hasn’t stopped deep learning engineers using them in their own servers.

There are also other options like using Google’s tensor processing units or buying pre-built machines, but these aren’t cheaper than building your own, Chen said. He also advised people to stick to Nvidia’s GPUs.

“Based what I’ve seen, [TPUs] don’t seem that much cheaper but promises to be much faster. I’ve looked into [cheaper alternatives to Nvidia] and decided against it. Most lack full software community support and AMD’s Vega is 25 to 50 per cent slower,” he told us. ®

We'll be examining machine learning, artificial intelligence, and data analytics, and what they mean for you, at Minds Mastering Machines in London, between October 15 and 17. Head to the website for the full agenda and ticket information.


Other stories you might like

  • Twitter founder Dorsey beats hasty retweet from the board
    As shareholders sue the social network amid Elon Musk's takeover scramble

    Twitter has officially entered the post-Dorsey age: its founder and two-time CEO's board term expired Wednesday, marking the first time the social media company hasn't had him around in some capacity.

    Jack Dorsey announced his resignation as Twitter chief exec in November 2021, and passed the baton to Parag Agrawal while remaining on the board. Now that board term has ended, and Dorsey has stepped down as expected. Agrawal has taken Dorsey's board seat; Salesforce co-CEO Bret Taylor has assumed the role of Twitter's board chair. 

    In his resignation announcement, Dorsey – who co-founded and is CEO of Block (formerly Square) – said having founders leading the companies they created can be severely limiting for an organization and can serve as a single point of failure. "I believe it's critical a company can stand on its own, free of its founder's influence or direction," Dorsey said. He didn't respond to a request for further comment today. 

    Continue reading
  • Snowflake stock drops as some top customers cut usage
    You might say its valuation is melting away

    IPO darling Snowflake's share price took a beating in an already bearish market for tech stocks after filing weaker than expected financial guidance amid a slowdown in orders from some of its largest customers.

    For its first quarter of fiscal 2023, ended April 30, Snowflake's revenue grew 85 percent year-on-year to $422.4 million. The company made an operating loss of $188.8 million, albeit down from $205.6 million a year ago.

    Although surpassing revenue expectations, the cloud-based data warehousing business saw its valuation tumble 16 percent in extended trading on Wednesday. Its stock price dived from $133 apiece to $117 in after-hours trading, and today is cruising back at $127. That stumble arrived amid a general tech stock sell-off some observers said was overdue.

    Continue reading
  • Amazon investors nuke proposed ethics overhaul and say yes to $212m CEO pay
    Workplace safety, labor organizing, sustainability and, um, wage 'fairness' all struck down in vote

    Amazon CEO Andy Jassy's first shareholder meeting was a rousing success for Amazon leadership and Jassy's bank account. But for activist investors intent on making Amazon more open and transparent, it was nothing short of a disaster.

    While actual voting results haven't been released yet, Amazon general counsel David Zapolsky told Reuters that stock owners voted down fifteen shareholder resolutions addressing topics including workplace safety, labor organizing, sustainability, and pay fairness. Amazon's board recommended voting no on all of the proposals.

    Jassy and the board scored additional victories in the form of shareholder approval for board appointments, executive compensation and a 20-for-1 stock split. Jassy's executive compensation package, which is tied to Amazon stock price and mostly delivered as stock awards over a multi-year period, was $212 million in 2021. 

    Continue reading
  • Confirmed: Broadcom, VMware agree to $61b merger
    Unless anyone out there can make a better offer. Oh, Elon?

    Broadcom has confirmed it intends to acquire VMware in a deal that looks set to be worth $61 billion, if it goes ahead: the agreement provides for a “go-shop” provision under which the virtualization giant may solicit alternative offers.

    Rumors of the proposed merger emerged earlier this week, amid much speculation, but neither of the companies was prepared to comment on the deal before today, when it was disclosed that the boards of directors of both organizations have unanimously approved the agreement.

    Michael Dell and Silver Lake investors, which own just over half of the outstanding shares in VMware between both, have apparently signed support agreements to vote in favor of the transaction, so long as the VMware board continues to recommend the proposed transaction with chip designer Broadcom.

    Continue reading
  • Perl Steering Council lays out a backwards compatible future for Perl 7
    Sensibly written code only, please. Plus: what all those 'heated discussions' were about

    The much-anticipated Perl 7 continues to twinkle in the distance although the final release of 5.36.0 is "just around the corner", according to the Perl Steering Council.

    Well into its fourth decade, the fortunes of Perl have ebbed and flowed over the years. Things came to a head last year, with the departure of former "pumpking" Sawyer X, following what he described as community "hostility."

    Part of the issue stemmed from the planned version 7 release, a key element of which, according to a post by the steering council "was to significantly reduce the boilerplate needed at the top of your code, by enabling a lot of widely used modules / pragmas."

    Continue reading

Biting the hand that feeds IT © 1998–2022