In September I presented at AWS CloudDay Warsaw 2024 where I delivered session DEV208 – Optimizing storage price and performance with Amazon S3. The event was held at PGE Narodowy on the 18th of September. Having spoken before in Warsaw back in 2022, it is always a pleasure to come back to this city and connect with the Polish tech community.
In this session I walked through how to get the most out of Amazon S3 from both a cost and performance perspective. We started with operating storage at scale and how to gain insights into your Amazon S3 usage using tools like Amazon S3 Storage Lens and S3 Inventory – helping you understand your storage patterns across accounts, regions, buckets and prefixes, and discover opportunities like incomplete multipart uploads and noncurrent object versions that might be costing you money.
I covered the full spectrum of Amazon S3 storage classes – from S3 Standard for frequently accessed data all the way down to S3 Glacier Deep Archive for long-term archival – and how to select the right one based on frequency of access, duration of storage and performance requirements. We also looked at S3 Lifecycle policies for data with known access patterns and Amazon S3 Intelligent-Tiering for data with unknown or changing access patterns, which automatically moves objects between access tiers to optimize costs without any operational overhead.
On the performance side, we discussed how Amazon S3 scales to tens of millions of requests per second and the key characteristics to consider: request rate, request latency and throughput. I shared practical tips on prefix design for bursty workloads, parallelizing operations using multipart uploads and byte-range fetches, and leveraging the AWS Common Runtime (CRT) which delivers up to 2.2x faster transfer speeds out of the box.
It was great to see so many people interested in Amazon S3 storage optimization!
My slides covering the presentation can be found here.