Amazon online storage

Increasingly we want to deliver high-bandwidth media from our websites.

But where do you keep all that data? ISPs and virtual hosting services typically base their pricing on capacity – and if you need upwards of 500MB it can be expensive.

Amazon’s S3 (Simple Storage Service) could provide an answer. It makes use of Amazon’s vast online data storage resources, hiring out spare capacity on a ‘pay for what you use’ basis. It’s fast, secure and cheap.

S3 is part of Amazon Web Services (AWS), a division of Amazon that devises ways to make parts of the company’s vast business infrastructure available to web developers on an affordable basis. Other Amazon Web Services include E-Commerce and the Electronic Compute Cloud. There’s more about these and other Amazon Web Services in the box on the next page.

About the only drawback of S3 is that it’s far from accessible, requiring knowledge of how to manipulate Soap (Simple Object Access Protocol) and Rest (Representational State Transfer) interfaces using Python, PHP or other languages. The good news is that applications that handle these kinds of tasks for you are now beginning to appear.

What it costs
S3 costs are based on usage – sign-up is free and you pay monthly data transfer fees for files uploaded to and downloaded from S3, as well as a monthly per gigabyte storage fee. The storage fee is $0.15 (about 8p) per GB-month. Until recently the ‘bandwidth’ price was an across-the-board $0.20 (about 10p) per GB for uploads and downloads.

As of 1 June a new pricing structure was introduced, with a $0.10 (about 5p) per GB upload fee and download fees on a reducing scale starting at $0.18 (about 9p) per GB for the first 10TB. An additional fee of $0.01 (about .5p) per 1,000 put or list requests and $0.01 (about .5p) per 10,000 get requests has also been introduced.

In practical terms what this means is that S3 is a very cost-effective way to deliver high-quality media files over the web.

Let’s say your site hosts 10GB of media and data download rates over a typical month are 6GB. To host this on S3 you’d pay a one-time upload fee of $1 (about 50p), a monthly storage fee of $1.50 (about 75p) and monthly download fees of $1.08 (about 54p). Assuming the get requests for these files totalled 20,000 it would only add another $0.02 (about 1p) to the bill. Your monthly S3 hosting bill would be $2.60 (about £1.30).

How it works
S3 isn’t a hosting provider in the conventional sense. It uses Soap and Rest protocols for transferring XML-based network messages using HTTP. For those used to hierarchical directory-based file structures, Amazon S3’s way of doing things may seem a little quirky. There are two basic storage concepts – objects and buckets. A bucket is like a directory on your hard drive and it contains objects – named blocks of data.

Objects can have different levels of access control, such as private or public read, which makes S3 secure enough to use as private backup as well as a web media server. You can also provide access on an individual authenticated user basis. AWS uses a public/private key pair for authentication enabling you to access your files as a means of identifying requests for security and billing purposes.

S3 URLs are of the form http://s3.amazonaws.com/[bucketname]/[objectname]. You can name a bucket whatever you like, but the name space is shared with everyone else using the service so you need to choose something unique. For a web media server the host site url combined with the filename provides a workable combination, for example mysite/mediafile.mov.