← Back to blog

The S3 Cost Optimization Playbook

awss3cost-optimizationcloud-infrastructure
Bar chart of S3 storage-class costs per GB-month, from S3 Standard to Glacier Deep Archive

Most S3 bills are wrong, and the fix takes an afternoon. The data sits in the most expensive class AWS offers (S3 Standard, $0.023/GB-month), nobody set a lifecycle policy, incomplete multipart uploads are silently billing for storage you can’t even see in the console, and every byte your EC2 fleet pulls from S3 is routed out through a NAT Gateway when a free VPC Gateway Endpoint would do the same job for $0. None of this needs an architecture rewrite. It needs a checklist run in the right order.

Here is the order. The savings depend entirely on your access pattern — I will not promise you a number I can’t see — but the mistakes below are so common that the question is usually how much, not whether. One number is just arithmetic: cold data that moves from S3 Standard ($0.023/GB-month) to Glacier Deep Archive ($0.00099/GB-month) drops about 96% on the storage line for those bytes, and on an observability platform I ran — logs aged past 90 days into Deep Archive — that is exactly the lever that did the work. S3 cost optimization is the same boring discipline as the rest of the bill: see it, then decide what each byte should actually cost.

First, see the bill before you touch it

You cannot optimize what you cannot measure, and S3’s default billing view tells you nearly nothing useful. Turn on S3 Storage Lens before anything else. The free tier gives you 62 metrics at the bucket level with 14 days of history, and crucially it includes cost-optimization metrics out of the box — including “Incomplete multipart upload bytes greater than 7 days old,” which is the single most common source of money disappearing into storage nobody knows exists (AWS S3 Storage Lens docs, accessed 2026-06-18).

Storage Lens free metrics answer the three questions that decide everything that follows:

  1. Which buckets hold the most bytes?
  2. What storage class is that data sitting in right now?
  3. Where are the incomplete multipart uploads?

For deeper per-prefix analysis or a longer history, Advanced metrics (15 months of data, recommendations) costs extra — worth it for a large estate, overkill for a single small account. Start free. Pay for advanced only once the free tier has told you the estate is big enough to justify it.

Pair this with S3 Storage Class Analysis on your busiest buckets. It watches access patterns and tells you which objects are candidates to move to Infrequent Access — so you set lifecycle thresholds from data, not from a guess.

Second, fix the silent leak: incomplete multipart uploads

This is the one nobody finds on their own, so it goes near the top.

When you upload a large object in parts and the upload fails partway — a dropped connection, a crashed job, an SDK that didn’t clean up — the parts that did land stay in the bucket. You are billed for that storage. They do not appear in the normal object listing. They accumulate for years, and on a long-lived account the orphaned parts can add up to a meaningful fraction of the bill before anyone notices.

The fix is one lifecycle rule, applied to every bucket:

{
  "Rules": [
    {
      "ID": "abort-incomplete-mpu",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

AWS supports a lifecycle rule that stops multipart uploads not completed within a set number of days and deletes the orphaned parts — and it applies to both existing and future uploads (AWS lifecycle config for incomplete MPU, accessed 2026-06-18). Seven days is the standard value; it is long enough that no legitimate in-flight upload gets killed, short enough that garbage doesn’t pile up. And per AWS, removing incomplete multipart parts via lifecycle does not trigger early-delete charges — so there is no downside.

Set this rule on every bucket you own, today, before you do anything else. It is the cheapest win in the whole playbook.

Third, get the data into the right storage class

S3 storage classes span a roughly 23x price range, and the only thing separating them is access pattern. Here is the ladder, US East (N. Virginia), per GB-month:

Class$/GB-monthMin durationMin object sizeUse it for
S3 Standard$0.023nonenoneActive, frequently read data
Standard-IA$0.012530 days128 KBRead a few times a month
One Zone-IA~$0.0130 days128 KBReproducible IA data, single-AZ OK
Glacier Instant Retrieval$0.00490 days128 KBArchive needing instant access
Glacier Flexible Retrieval$0.003690 days—Archive, minutes-to-hours retrieval
Glacier Deep Archive~$0.00099180 days—Compliance, rarely-ever read

Storage prices and the minimum-duration / minimum-size rules are from AWS primary docs (storage classes, Glacier classes, accessed 2026-06-18). Two rules carry most of the risk:

  • Minimum storage duration. Delete a Standard-IA object before 30 days and you still pay the full 30. Glacier Flexible and Instant bill a 90-day minimum; Deep Archive bills 180. Move data down the ladder only when it will actually sit there. Putting short-lived data in Glacier is a way to pay more, not less.
  • Minimum billable object size. Standard-IA, One Zone-IA, and Glacier Instant bill every object as if it were at least 128 KB. A bucket of 10 KB thumbnails moved to Standard-IA gets billed at 128 KB each — you pay for 12x the bytes you store. Small objects stay in Standard. This is the trap that quietly reverses the savings.

The decision rule is simple. Frequently read → Standard. Read occasionally, objects over 128 KB → Standard-IA. Rarely read but must be instant → Glacier Instant Retrieval. Archive you can wait minutes-to-hours for → Glacier Flexible. Compliance data you’ll likely never read → Deep Archive.

Fourth, automate the transitions with lifecycle policies

You do not move data by hand. You write a lifecycle policy and S3 does it on a schedule. A typical policy for log or backup data:

{
  "Rules": [
    {
      "ID": "tier-down-logs",
      "Status": "Enabled",
      "Filter": { "Prefix": "logs/" },
      "Transitions": [
        { "Days": 30,  "StorageClass": "STANDARD_IA" },
        { "Days": 90,  "StorageClass": "GLACIER_IR" },
        { "Days": 180, "StorageClass": "DEEP_ARCHIVE" }
      ],
      "Expiration": { "Days": 2555 }
    }
  ]
}

Two things to know before you ship a lifecycle policy:

  • Transitions cost money per request. Each lifecycle transition is a billed request, and the per-1,000 transition cost is higher for the colder classes (vendor: AWS S3 pricing — confirm the exact us-east-1 cents on the live page, the table is JS-rendered and these move). For a bucket of millions of tiny objects, the transition requests can cost more than the storage you save. This is the second reason small objects don’t belong in IA — the move itself isn’t worth it.
  • Expiration is the most underused line. If the data has a legal or practical end-of-life, set Expiration. Storage you delete is storage you stop paying for forever. Most teams tier data down and then keep it for eternity because nobody wrote the expiry rule.

Fifth, when access is unpredictable, use Intelligent-Tiering

Lifecycle policies assume you know the access pattern. When you don’t — user uploads, a data lake, anything where some objects go cold and others stay hot unpredictably — S3 Intelligent-Tiering is the right default. It monitors each object and moves it between tiers automatically: after 30 consecutive days with no access it drops to Infrequent Access (about 40% cheaper), and after 90 days to Archive Instant Access (about 68% cheaper), with no retrieval fee when an object gets hot again and is read (AWS Intelligent-Tiering, accessed 2026-06-18).

The cost is a monitoring-and-automation charge of $0.0025 per 1,000 objects per month (vendor: AWS, us-east-1). That math has one sharp edge:

  • Intelligent-Tiering is wrong for billions of tiny objects. The monitoring fee is per object, not per GB. A bucket of a billion small objects pays a monitoring charge that can dwarf any tiering savings. Per AWS, objects smaller than 128 KB are never auto-tiered and are always billed at the Frequent Access rate — but they can still rack up monitoring charges. For huge counts of small objects, a plain lifecycle policy (or just Standard) beats Intelligent-Tiering.

Rule of thumb: unknown access pattern + reasonably sized objects → Intelligent-Tiering and forget it. Known pattern, or billions of tiny objects → explicit lifecycle policy.

Sixth, stop paying for retrieval and requests you didn’t budget for

Storage is the line everyone watches. Requests and retrievals are the lines that ambush you.

  • Retrieval fees scale with how cold the class is. Standard-IA and Glacier Instant charge a per-GB read fee; Glacier Flexible and Deep Archive charge a per-GB retrieval fee plus a per-request fee, and Deep Archive’s slowest retrieval tier is measured in hours, not seconds (vendor: AWS — confirm the exact per-GB cents and the retrieval-time SLA on the live S3 pricing and retrieval-options pages). The lesson: a class is only cheap if you read it as rarely as its design assumes. Putting frequently-read data in Glacier to save on storage and then paying retrieval on every read is the most expensive mistake in this whole document — the retrieval bill can exceed what Standard would have cost outright.
  • Request pricing punishes chatty workloads. S3 Standard GETs are cheap individually (~$0.0004 per 1,000) but a service doing millions of tiny GETs per minute turns “cheap” into a real line item. Batch, cache, and use CloudFront in front of read-heavy buckets so the requests never hit S3.

Before you tier anything down, ask the one question that governs the whole ladder: how often is this actually read? If you don’t know, that’s what Storage Class Analysis and Intelligent-Tiering are for. If you guess wrong toward “cold,” the retrieval fees make you pay for the guess.

Seventh, the free win everyone leaves on the table: VPC Gateway Endpoints

If your EC2, ECS, or Lambda workloads in a VPC talk to S3, check how that traffic leaves the VPC. By default, instances in a private subnet reach S3 through a NAT Gateway — and NAT Gateway bills both an hourly charge and a per-GB data-processing charge on every byte. For a workload pulling terabytes from S3, that is a tax you are paying for nothing. (The full version of that problem is its own post: the NAT Gateway hidden tax.)

An S3 Gateway VPC Endpoint routes that traffic privately, and AWS charges nothing for it — no hourly fee, no per-GB fee, and traffic to S3 in the same Region incurs no data transfer charge (AWS Gateway endpoints for S3, accessed 2026-06-18). You add a route, and the same S3 traffic that was flowing through a metered NAT Gateway now flows free.

One caveat worth stating honestly: Gateway endpoints work for traffic originating inside the VPC in the same Region. They do not serve on-premises networks, peered VPCs in other Regions, or transit-gateway paths — those need an Interface endpoint, which does cost money ($0.01/AZ/hour plus $0.01/GB). For the common case — instances in a VPC reading from S3 in the same Region — the Gateway endpoint is free and you should have created it on day one.

The order matters

Run it top to bottom: turn on Storage Lens, kill incomplete multipart uploads, right-size storage classes, automate with lifecycle (or Intelligent-Tiering when the pattern is unknown), respect the retrieval and request fees, and add the free VPC Gateway Endpoint. Every step is reversible, none of it touches your application code, and the whole thing is an afternoon’s work for savings that compound every month the data sits there.

The reason this works is not cleverness. It is that S3’s defaults are tuned for the most expensive, most available configuration, and almost nobody changes them. The money is sitting in plain sight. You just have to run the checklist.

Sources