For DevOps engineers and engineering leads responsible for cloud storage cost management at Indian businesses.
What Lifecycle Policies Do
S3 lifecycle policies are rules that automatically act on objects in a bucket based on their age, prefix, or tags. They can delete objects after a specified number of days, transition objects to cheaper storage classes, expire previous object versions, and clean up incomplete multipart uploads.
Lifecycle policies require no application code changes. They run server-side on a schedule set by the rule. Once configured, they reduce storage costs automatically every day without requiring anyone to remember to run a cleanup job.
For most Indian engineering teams, implementing the six policies described in this guide will reduce the cloud storage bill by 20–40% within the first billing cycle.
How to Apply Policies on S3-Compatible Storage
Lifecycle policies are applied to buckets via the S3 API. Using the AWS CLI:
The --endpoint-url flag points the CLI at IBEE. On AWS S3, omit this flag. The lifecycle JSON format is identical.
Policy 1 — Abort Incomplete Multipart Uploads
Every team that accepts file uploads has this cost lurking in their bill. When a multipart upload is initiated but not completed — because of a client error, a network interruption, or a crash — the uploaded parts remain in storage and continue to be billed indefinitely. They are invisible in normal bucket listings.
This policy cleans up incomplete multipart uploads after 7 days:
This is the highest-priority lifecycle policy because the savings are pure waste elimination — no application changes, no data loss risk. Apply it to every bucket that accepts uploads.
Policy 2 — Expire Application Logs After 90 Days
Application logs, access logs, and audit logs accumulate without bound if no expiry policy is set. For most Indian businesses, the relevant regulatory requirement is CERT-In's 180-day retention mandate for IT system logs. Beyond that, most logs have diminishing value after 30–90 days.
Adjust the prefix to match your log storage structure. For CERT-In-regulated entities, set the expiry to 180 days (not less). For application debug logs with no regulatory requirement, 30 days is typically sufficient.
Policy 3 — Clean Up Old CI/CD Build Artifacts
CI/CD pipelines write build artifacts, test reports, and deployment packages to S3 on every run. Without an expiry policy, these accumulate indefinitely. A team running 50 builds per day, each producing 200 MB of artifacts, generates 3 TB of artifact storage per month.
Retain only as many days of build artifacts as your incident response and rollback process requires. Most teams need 7–14 days. Thirty days is conservative.
Policy 4 — Expire Temporary Processing Files
Data pipelines, image processing queues, transcoding workflows, and batch jobs often create intermediate files in S3. These temporary files should be cleaned up after the processing pipeline completes, but if the cleanup step is skipped due to a pipeline failure, they accumulate.
Organise all temporary pipeline output under a consistent prefix (tmp/, processing/, staging/) so the lifecycle rule covers it with a single prefix filter.
Policy 5 — Expire Non-Current Object Versions
If bucket versioning is enabled (which it should be on important data buckets), every overwrite creates a new version and retains the previous version. Without a version expiry policy, previous versions accumulate indefinitely.
This retains the previous 30 days of versions (for recovery from accidental overwrites) and deletes older versions automatically. For buckets with high write frequency, this policy can dramatically reduce storage costs.
Policy 6 — Delete Expired Delete Markers
When a versioned bucket has delete markers (the result of deleting an object in a versioned bucket), and all versions of that object have been expired, the delete marker becomes an "expired delete marker" — it still occupies a listing entry but no object content. These accumulate silently.
This is a housekeeping policy with marginal cost impact but meaningful operational hygiene — it prevents bucket listings from accumulating thousands of phantom delete marker entries.
Combining Policies in One Configuration
Multiple rules can be combined in a single lifecycle configuration. A comprehensive policy for a production application bucket:
For log and artifact buckets, add the relevant expiry rule for that bucket's specific data type.
IBEE and Lifecycle Policies
IBEE's S3-compatible storage supports lifecycle policy configuration through the standard S3 API. All six policies described here apply using the same JSON format and AWS CLI commands, with the --endpoint-url flag pointing at IBEE.
IBEE's flat pricing model without retrieval surcharges means you do not need to manage tier transitions — the single storage class is already priced at Rs.1.50/GB-month with no retrieval penalties. On AWS, lifecycle tier transitions involve additional complexity around retrieval fees and minimum storage durations per tier. On IBEE, lifecycle policies simplify to: keep what you need, expire what you do not.
