tl;dr: The Crazy Scale of Amazon S3

There's a pretty cool article about the scale of S3: Building and operating a pretty big storage system called S3.

It's pretty long: ~6.5k words, ~36k chars. Here's the tl;dr version of it.


280 TRILLION objects, with a T! 4 billion checksums per second! MILLIONS of hard disks!

Holy fucking shit.

I suspected Amazon dealt with numbers like these, considering it pretty much hosts a large chunk of the entire internet, but seeing these numbers feels unreal.

Hard disks bit error rate

Hard disks have a bit error rate of 1 in 10^15 requests. Mortal humans usually don't even need to think about this. S3 actually encounters this pretty frequently and needs to account for it.

How to deal with eventual consistency

Don't. Just let there be inconsistency — it isn't a big deal.

Dynamo was eventually consistent, so it was possible for your shopping cart to be wrong. [...] ultimately, these conflicts were rare, and you could resolve them by getting support staff involved and making a human decision.

This is a key takeaway for me. If the problem is small enough at Amazon's scale, it's something I generally needn't worry about.

Leadership, Motivation, Ownership

Explain what the problem is and let people come up with their own solutions.

It’s a lot harder to get invested in an idea that you don’t own. I consciously spend a lot more time trying to develop problems, and to do a really good job of articulating them, rather than trying to pitch solutions.

Super biased and with a ton of caveats, but the idea is interesting.

A newsletter for programmers

Yo! This is Taro. I've been doing JavaScript for years and TypeScript for years. I have experience with many programming languages, libraries, frameworks; both backend and frontend, and in a few company roles/positions.

I learned a few things over the years. Some took more effort than I wish they had. My goal with this blog and newsletter is to help frontend and backend developers by sharing what I learned in a friendlier, more accessible and thorough manner.

I write about cool and new JavaScript, TypeScript and CSS features, architecture, the human side of working in IT, my experience and software-related things I enjoy in general.

Subscribe to my newsletter to receive notifications when I publish new articles, as well as some newsletter-exclusive content.

No spam. Unsubscribe at any time. I'll never share your details with anyone. 1 email a week at most.

You have subscribed to Taro's newsletter
The server blew up. I'll go get my fire extinguisher — please check back in 5.