Amazon Simple Storage Service (S3) is a highly durable object storage service and its HTTP REST API has become the de facto standard for cloud object storage, widely re-implemented by other vendors.

What is S3? Definition, examples and how it works

S3 (Simple Storage Service) is an object storage service introduced by Amazon Web Services in 2006. It stores data as objects within buckets, where each object consists of data, a key, and metadata. Objects are identified by a unique key within a bucket, and access is controlled via bucket policies and IAM roles. The service offers 99.999999999% durability (11 nines) by replicating data across multiple availability zones.

The S3 API is a RESTful HTTP interface that uses standard HTTP methods (GET, PUT, DELETE, HEAD) to manipulate objects and buckets. Requests are authenticated using AWS Signature Version 4, which signs each request with a secret key. The API supports range reads, multipart uploads for large objects, and server-side encryption. S3 provides both strong read-after-write consistency for PUT operations of new objects and eventual consistency for overwrite PUTs and DELETE operations (though as of 2020, Amazon claims strong consistency for all operations in most regions).

S3's API has become the de facto standard for object storage. Many third-party services, including MinIO, Ceph, DigitalOcean Spaces, and Google Cloud Storage (via interoperability mode), implement the S3 API to ensure compatibility. This ubiquity allows applications written for AWS S3 to work with alternative storage backends with minimal changes. The API is also used by backup tools, media workflows, and data lakes. Despite its dominance, the S3 API is not an official standard; it is a proprietary API that has been reverse-engineered and re-implemented by the open-source community and other cloud providers.

Key facts

Launched by Amazon in 2006; now the most widely used object storage API.
Uses a flat namespace of buckets and objects, each identified by a key.
Offers 99.999999999% durability through automatic replication across facilities.
RESTful API over HTTP with AWS Signature V4 for authentication.
Re-implemented by MinIO, Ceph, DigitalOcean, Google Cloud, and many others.

How it works in practice

A media company stores video files in an S3 bucket. A user uploads a 2 GB file via the multipart upload API, which splits the file into 5 MB parts. Each part is uploaded independently and then assembled. The final object is stored with custom metadata (e.g., resolution, codec). A CDN pulls the object using the S3 API's GET request, serving it to viewers worldwide. The company later migrates to a MinIO cluster on-premises, and the application continues to work because MinIO implements the same S3 API.

Related terms

Object storage Bucket AWS Signature Version 4 MinIO Ceph Multipart upload REST API

References

More in Storage

Block Storage

Block storage is a storage architecture that presents raw fixed-size chunks (blocks) to a computer, which the operating system partitions and formats before use, commonly used for virtual machine disks and database volumes.

Cold Storage

Cold storage is a low-cost, slow-retrieval data tier for infrequently accessed information, such as backups, archives, or compliance records, with retrieval times ranging from minutes to hours.

Erasure Coding

Erasure coding is a data protection method that splits data into fragments (shards) and adds parity, allowing reconstruction from a subset of fragments; it is commonly used in cloud object storage to tolerate failures efficiently.

Fibre Channel

Fibre Channel is a high-speed networking standard primarily used to connect storage devices in storage area networks, operating as a dedicated protocol stack separate from Ethernet.

File Storage

File storage exposes data as a hierarchical filesystem over network protocols like NFS and SMB, commonly delivered via network-attached storage (NAS) appliances for shared access.

Hot Storage

Hot storage is the default high-performance tier in object storage systems, optimized for low-latency access to frequently retrieved data.

iSCSI

iSCSI is a storage protocol that encapsulates SCSI block-level commands over TCP/IP, enabling servers to access remote disks as if they were locally attached.

NFS

NFS (Network File System) is a distributed file system protocol originally developed by Sun Microsystems that allows a client to access files over a network as if they were local. It is widely used in Unix and Linux environments and runs over IP networks.

Object Storage

Object storage is a flat data storage architecture that manages data as immutable objects, each with a unique identifier, metadata, and API access over HTTP. It is the model behind Amazon S3 and similar cloud storage services.

RAID

RAID (Redundant Array of Independent Disks) combines multiple physical disk drives into a single logical unit to improve performance, fault tolerance, or both, using techniques like striping, mirroring, and parity.