Amazon's S3 object count kisses 1 trillion

Buying Bezos by slice


It started out so small, a mere 200,000 objects, with the 2006 launch of the Simple Storage Service (S3) object storage companion to the Elastic Compute Cloud (EC2) compute cloud. Six years later, and Amazon's infrastructure cloud is getting ready to handily bust through 1 trillion objects under management.

In a blog post, Jeff Barr, at the Amazon Web Services infrastructure unit, says that the S3 service contained 905 billion objects as of the end of the first quarter. This is an amazing number of files for any single entity to house.

Perhaps more significantly, the peak number of requests that the S3 service is fielding is now at 650,000 requests per second, with occasional peaks going even higher.

Amazon has kept the S3 data a little close to its chest over the years, but here is what the Q1 2012 data looks like against the data from the fourth quarter from each year since the S3 service was launched:

Amazon AWS S3 Q4 data

Amazon's S3 object growth over time (click to enlarge)

El Reg has poked around gathering up other data points here and there over the past year, and we have assembled more quarterly data about the S3 service than Barr put in his blog. Every point with a blue diamond is an actual data point for that quarter in the chart below. We fit the curve as best we could between the dots.

Amazon AWS S3 objects

The S3 cloud: Not quite a hockey stick – yet

The tapering off of the S3 object growth in the first quarter was no doubt in part due to the launch of object expiration and multi-object deletion, both of which came out in December 2011.

Barr says that every day AWS import/export, the S3 storage gateway, the S3 APIs, Direct Connect pipes, and various backup tools that work with S3 are adding "well over a billion objects" per day.

Actually, it is closer to 1.57 billion a day in the first quarter of this year, if you do the math on the Q4 2011 and Q1 2012 data and average over the days.

In a sense, by offering the automatic file retirement services, Amazon chopped off the handle of this hockey stick curve. This may make the S3 service look less impressive in terms of straight math but it also makes S3 a more useful service and hence more appealing to customers who shell out bucks to use Amazon's disk drives instead of their own.

Amazon has not given out peak file request data as frequently as the object count, but this is what we know. Back in the first quarter of 2009, Amazon said S3 was peaking at around 70,000 file requests per second, and in some customer presentations Amazon said it hit 200,000 peak requests per second in the second quarter of 2011.

We were told earlier this year when we did a story on the S3 ramp that it was peaking at 370,000 requests per second, and now it has rocketed up to 650,000 requests per second.

So even as the service is now automagically retiring files to the bit bucket, the amount of requests is on a very fast ramp indeed. It would be interesting to see what the bandwidth is on the S3 cloud, and how much money it generates.

Show me the money

And where is the comparable EC2 data?

Amazon does not, of course, talk about the dough that AWS generates in the aggregate or for any particular service individually.

In the first nine months of 2011, Amazon posted $30.6bn in sales (up 44.2 per cent) while the Other category, where AWS and various other unspecified product sales get parked in Amazon's financials, accounted for $1.08bn in sales (up 70.4 per cent).

Assume that when Amazon reports its final 2011 quarter the Other category is around $1.5bn, and maybe three quarters of the revenue is AWS. If those assumptions are correct, then AWS has already broken through $1bn in annual sales.

If that were the case, AWS would still only be about three per cent of revenue, but how many IT departments in the world are making money instead of spending money?

The interesting question is this: Are the millions of EC2, S3, and other AWS service users now giving Amazon enough money to use its data center to pay for Amazon's own usage out of the profits? ®

Similar topics


Other stories you might like

  • New audio server Pipewire coming to next version of Ubuntu
    What does that mean? Better latency and a replacement for PulseAudio

    The next release of Ubuntu, version 22.10 and codenamed Kinetic Kudu, will switch audio servers to the relatively new PipeWire.

    Don't panic. As J M Barrie said: "All of this has happened before, and it will all happen again." Fedora switched to PipeWire in version 34, over a year ago now. Users who aren't pro-level creators or editors of sound and music on Ubuntu may not notice the planned change.

    Currently, most editions of Ubuntu use the PulseAudio server, which it adopted in version 8.04 Hardy Heron, the company's second LTS release. (The Ubuntu Studio edition uses JACK instead.) Fedora 8 also switched to PulseAudio. Before PulseAudio became the standard, many distros used ESD, the Enlightened Sound Daemon, which came out of the Enlightenment project, best known for its desktop.

    Continue reading
  • VMware claims 'bare-metal' performance on virtualized GPUs
    Is... is that why Broadcom wants to buy it?

    The future of high-performance computing will be virtualized, VMware's Uday Kurkure has told The Register.

    Kurkure, the lead engineer for VMware's performance engineering team, has spent the past five years working on ways to virtualize machine-learning workloads running on accelerators. Earlier this month his team reported "near or better than bare-metal performance" for Bidirectional Encoder Representations from Transformers (BERT) and Mask R-CNN — two popular machine-learning workloads — running on virtualized GPUs (vGPU) connected using Nvidia's NVLink interconnect.

    NVLink enables compute and memory resources to be shared across up to four GPUs over a high-bandwidth mesh fabric operating at 6.25GB/s per lane compared to PCIe 4.0's 2.5GB/s. The interconnect enabled Kurkure's team to pool 160GB of GPU memory from the Dell PowerEdge system's four 40GB Nvidia A100 SXM GPUs.

    Continue reading
  • Nvidia promises annual updates across CPU, GPU, and DPU lines
    Arm one year, x86 the next, and always faster than a certain chip shop that still can't ship even one standalone GPU

    Computex Nvidia's push deeper into enterprise computing will see its practice of introducing a new GPU architecture every two years brought to its CPUs and data processing units (DPUs, aka SmartNICs).

    Speaking on the company's pre-recorded keynote released to coincide with the Computex exhibition in Taiwan this week, senior vice president for hardware engineering Brian Kelleher spoke of the company's "reputation for unmatched execution on silicon." That's language that needs to be considered in the context of Intel, an Nvidia rival, again delaying a planned entry to the discrete GPU market.

    "We will extend our execution excellence and give each of our chip architectures a two-year rhythm," Kelleher added.

    Continue reading
  • Amazon puts 'creepy' AI cameras in UK delivery vans
    Big Bezos is watching you

    Amazon is reportedly installing AI-powered cameras in delivery vans to keep tabs on its drivers in the UK.

    The technology was first deployed, with numerous errors that reportedly denied drivers' bonuses after malfunctions, in the US. Last year, the internet giant produced a corporate video detailing how the cameras monitor drivers' driving behavior for safety reasons. The same system is now apparently being rolled out to vehicles in the UK. 

    Multiple camera lenses are placed under the front mirror. One is directed at the person behind the wheel, one is facing the road, and two are located on either side to provide a wider view. The cameras are monitored by software built by Netradyne, a computer-vision startup focused on driver safety. This code uses machine-learning algorithms to figure out what's going on in and around the vehicle.

    Continue reading
  • AWS puts latest homebrew ‘Graviton 3’ Arm CPU in production
    Just one instance type for now, but cheaper than third-gen Xeons or EPYCs

    Amazon Web Services has made its latest homebrew CPU, the Graviton3, available to rent in its Elastic Compute Cloud (EC2) infrastructure-as-a-service offering.

    The cloud colossus launched Graviton3 at its late 2021 re:Invent conference, revealing that the 55-billion-transistor device includes 64 cores, runs at 2.6GHz clock speed, can address DDR5 RAM and 300GB/sec max memory bandwidth, and employs 256-bit Scalable Vector Extensions.

    The chips were offered as a tech preview to select customers. And on Monday, AWS made them available to all comers in a single instance type named C7g.

    Continue reading
  • Beijing reverses ban on tech companies listing offshore
    Announcement comes as Chinese ride-hailing DiDi Chuxing delists from NYSE under pressure

    The Chinese government has announced that it will again allow "platform companies" – Beijing's term for tech giants – to list on overseas stock markets, marking a loosening of restrictions on the sector.

    "Platform companies will be encouraged to list on domestic and overseas markets in accordance with laws and regulations," announced premier Li Keqiang at an executive meeting of China's State Council – a body akin to cabinet in the USA or parliamentary democracies.

    The statement comes a week after vice premier Liu He advocated technology and government cooperation and a digital economy that supports an opening to "the outside world" to around 100 members of the Chinese People's Political Consultative Congress (CPPCC).

    Continue reading
  • Nvidia teases server designs for Grace-Hopper Superchips
    x86 still 'very important' we're told as lid lifted on Arm-based kit

    Computex Nvidia's Grace CPU and Hopper Superchips will make their first appearance early next year in systems that'll be based on reference servers unveiled at Computex 2022 this week.

    It's hoped these Arm-compatible HGX-series designs will be used to build computer systems that power what Nvidia believes will be a "half trillion dollar" market of machine learning, digital-twin simulation, and cloud gaming applications.

    "This transformation requires us to reimagine the datacenter at every level, from hardware to software from chips to infrastructure to systems," Paresh Kharya, senior director of product management and marketing at Nvidia, said during a press briefing.

    Continue reading
  • Nvidia brings liquid cooling to A100 PCIe GPU cards for ‘greener’ datacenters
    For those who want to give their racks an air cut

    Nvidia's GPUs are becoming increasingly more power hungry, so the US giant is hoping to make datacenters using them "greener" with liquid-cooled PCIe cards that contain its highest-performing chips.

    At this year's Computex event in Taiwan, the computer graphics goliath revealed it will sell a liquid-cooled PCIe card for its flagship server GPU, the A100, in the third quarter of this year. Then in early 2023, the company plans to release a liquid-cooled PCIe card for the A100's recently announced successor, the Hopper-powered H100.

    Nvidia's A100 has already been available for liquid-cooled servers, but to date, this has only been possible in the GPU's SXM form factor that goes into the company's HGX server board.

    Continue reading

Biting the hand that feeds IT © 1998–2022