Won't duke, duke, duke the URLs: AWS backtracks on plans to block old-style S3 paths

As I walk through this world... born-in-the-cloud companies have legacy problems too

10 Reg comments Got Tips?

Amazon Web Services has changed its mind about ending support for the URLs originally used by S3 (Simple Storage Service), though it is delaying rather than ending its deprecation plans.

S3, introduced in 2006, is one of the oldest AWS services. It is cloud storage where files can be accessed programmatically or published to the internet, this latter feature being a common cause of unintended data leaks.

In its original design, all S3 URLs began with an S3 domain such as s3.amazonaws.com or s3-eu-west-1.amazonaws.com. AWS calls these path-style addresses. This was later changed to support addresses where each "bucket", or container for files, is a subdomain. This type of URL begins yourdomain.s3.amazonaws.com and AWS calls it virtual hosted addressing. For example:

Old path style:

https://s3.amazonaws.com/TestBucket/MyFile.txt

New virtual hosted style:

https://TestBucket.s3-eu-west-1.amazonaws.com/MyFile.txt

Currently, buckets support both styles so the above examples could point to the same file.

AWS would like to remove support for path-style URLs. S3 has "many trillions of objects and processes millions of requests per second for them," according to its post on the subject.

The path-style URLs means that all these requests go to a limited number of endpoints, whereas with the virtual hosted style, each bucket has its own entry in DNS, the distributed database through which the internet maps names to locations.

AWS said the virtual hosted style makes scaling easier as well as helping it to defend against DDoS (Distributed Denial of Service) attacks. Some new security features depend on using virtual hosted addresses.

There are complications, though. Domains and subdomains are not case-sensitive, but paths (everything after the double slash) potentially are. There are also issues with SSL (secure sockets layer) and wildcard certificates if there are dots in the bucket name. A wildcard certificate may match foo.somedomain.com but not foo.bar.somedomain.com.

These dotted bucket names are fine in the path-style URL, but AWS currently does not support them in the virtual hosted style. Another issue is that some characters are valid in the path but not in the domain name.

AWS did plan to end support for path-style URLs completely on September 30. Then it said, after protests from customers with applications that depend on them, that it would continue to support path style for existing buckets, but not for new ones created after the deadline.

Now it is has backtracked further, saying that it heard from "many customers who have asked us to extend the deprecation date". AWS said that it will postpone the deprecation to "ensure that customers have the time they need to transition to virtual hosted-style URLs."

No time scale is given, but there is a clue based on the dot-in-bucket issue. AWS said it is working on providing support for dotted bucket names, and that once it delivers that support, there will be at least a year before path-style URLs are no longer supported.

Ending path-style URLs for new buckets is not so bad, though developers may have to revise their code. Ending path-style support for existing buckets is more serious, because it breaks URLs for existing files, adding to the multitude of broken links that already afflict the internet as well as potentially breaking applications. ®


Biting the hand that feeds IT © 1998–2020