Amazon S3 Tables Now Fully Support Apache Iceberg V3 to Transform Large-Scale Enterprise Analytics

The modern data architecture landscape is undergoing a significant evolution as cloud storage and open table formats converge to solve the performance bottlenecks of petabyte-scale analytics. Amazon Web Services (AWS) has announced full support for all data types and advanced capabilities outlined in the Apache Iceberg V3 specification within Amazon S3 Tables. This development allows organizations managing massive data lakes to transition away from legacy workarounds, complex string-encoding schemes, and resource-intensive positional delete files. By integrating the newest features of the open standard—such as native variant types, deletion vectors, and built-in row lineage—AWS aims to streamline data pipelines, reduce cloud storage costs, and improve query performance for global enterprises operating across multi-engine environments.

The evolution of open table formats has redefined how organizations approach data lakes, shifting the paradigm from rigid proprietary warehouses to flexible, standards-compliant storage layers. Apache Iceberg emerged as a premier solution for managing large datasets on object storage, allowing enterprises to maintain petabyte-scale tables using open Parquet files. However, as data volumes scaled into the tens of billions of rows, engineering teams running workloads on Iceberg V2 tables frequently encountered architectural limitations. Routine operations—such as executing compliance requests to delete specific user records from multi-billion-row repositories—generated fragmented positional delete files. These files degraded query efficiency until expensive background compaction processes could be executed. Furthermore, incoming semi-structured data streams, such as JSON events, geospatial coordinates, and high-precision timestamps, often required costly string encoding or custom parsing routines that increased operational overhead and query latency.

To address these industry-wide friction points, the Apache Software Foundation developed the V3 specification, introducing structural enhancements designed to optimize row-level updates, data governance, and semi-structured data management. Capitalizing on this open-source advancement, Amazon S3 Tables—a purpose-built storage tier optimized to maintain high performance and cost-effectiveness for Iceberg tables—has implemented comprehensive support for the V3 standard. Organizations can now provision new V3 tables or execute in-place upgrades of existing V2 assets. Fully managed AWS features, including automatic compaction, routine maintenance, replication, and Intelligent-Tiering, continue to operate seamlessly behind the scenes, insulating data engineering teams from manual infrastructure tuning.

At the core of the Apache Iceberg V3 specification are several critical enhancements designed to eliminate legacy workflow inefficiencies. Chief among these is the introduction of deletion vectors, which replace V2 positional delete files with a compact, binary-encoded format. Instead of generating thousands of fragmented delete files when purging records—such as fulfilling regulatory compliance mandates to remove thousands of user profiles from massive datasets—the engine now writes a single, streamlined deletion vector file. This architectural shift drastically minimizes compaction times and eliminates delete file overhead. Additionally, V3 introduces native row lineage capabilities by automatically appending unique identifiers (_row_id) and sequence numbers (_last_updated_sequence_number) to every record. Downstream data pipelines can leverage these metadata fields to isolate and ingest modified rows directly, bypassing the need to perform costly, full-table scans during incremental data ingestion tasks.

Beyond operational updates, the V3 specification introduces native data types that eliminate the need for cumbersome string or integer conversions. The new variant data type, for instance, allows teams to store semi-structured data in an optimized columnar format. During write operations, the processing engine shreds variant payloads into hidden columns while collecting statistical metadata. When executing analytical queries, these statistics enable advanced file pruning, significantly lowering input/output (I/O) costs compared to traditional methods of parsing raw JSON strings at read time. Furthermore, the inclusion of native support for nanosecond-precision timestamps, geometry, and geography data types empowers industries such as retail, telecommunications, and geospatial mapping to store and query complex domain-specific information natively within their primary analytical repositories.

In practical application, the integration of these capabilities transforms how data engineering and retail analytics teams architect their data pipelines. Consider a multi-channel retail enterprise tracking millions of customer interaction events across web and mobile platforms. Historically, accommodating diverse event payloads—such as page views containing URLs and durations, purchase logs detailing items and transaction amounts, and search events recording query terms—required complex schema evolution strategies or normalized relational designs. With S3 Tables and Apache Iceberg V3, engineers can define a single unified table utilizing the variant data type without enforcing a rigid upfront schema:

CREATE TABLE my_catalog.namespace.clickstream (
event_id bigint,
event_time timestamp,
user_id string,
payload variant
)
USING iceberg
TBLPROPERTIES (‘format-version’ = ‘3’)

Data ingestion workflows can subsequently stream heterogeneous event payloads directly into the table without triggering schema migration alerts. For example, insert operations can handle divergent JSON structures natively:

INSERT INTO my_catalog.namespace.clickstream VALUES
(1, current_timestamp(), ‘user-42’,
PARSE_JSON(‘"action": "purchase", "amount": 99.99, "items": ["laptop_stand"]’)),
(2, current_timestamp(), ‘user-17’,
PARSE_JSON(‘"action": "page_view", "url": "/products/webcam", "duration_ms": 4200’));

When querying this data using analytics engines such as Amazon EMR Spark, analysts can query the variant column directly using specialized functions like variant_get, eliminating the performance penalty associated with real-time JSON parsing:

SELECT
event_id,
user_id,
variant_get(payload, ‘$.action’, ‘string’) AS action,
variant_get(payload, ‘$.amount’, ‘double’) AS amount
FROM my_catalog.namespace.clickstream
WHERE variant_get(payload, ‘$.action’, ‘string’) = ‘purchase’
AND variant_get(payload, ‘$.amount’, ‘double’) > 50.00

To harness the performance benefits of deletion vectors during write and modification cycles, administrators can configure the table properties to operate in a merge-on-read mode:

ALTER TABLE my_catalog.namespace.clickstream
SET TBLPROPERTIES (
‘write.delete.mode’ = ‘merge-on-read’,
‘write.update.mode’ = ‘merge-on-read’,
‘write.merge.mode’ = ‘merge-on-read’
)

Executing compliance-related data purges under this configuration no longer triggers extensive data file rewrites. Instead, the database engine writes a lightweight deletion vector:

DELETE FROM my_catalog.namespace.clickstream
WHERE user_id = ‘user-42’

The background maintenance

routines native to Amazon S3 Tables automatically process and resolve these deletion vector files during scheduled maintenance windows, ensuring optimal storage hygiene without manual administrative intervention.

For organizations managing existing data assets, AWS has engineered a streamlined, backwards-compatible upgrade path from Iceberg V2 to V3. Enterprises can execute an in-place table upgrade atomically, eliminating the need to move or rewrite existing data files:

ALTER TABLE my_catalog.namespace.existing_table
SET TBLPROPERTIES (‘format-version’ = ‘3’)

During subsequent maintenance cycles, S3 Tables systematically purge legacy V2 delete files, while newly executed modifications automatically adopt deletion vectors and initialize row lineage tracking. Industry analysts note that because upgrading to V3 is an irreversible, one-way specification operation, data platform architects must verify that all downstream reading engines and analytics tools across their enterprise ecosystem maintain full V3 compatibility before initiating production upgrades. To maintain operational continuity during multi-phase migrations, AWS services ensure that legacy V2 readers can continue to access upgraded tables until engineering teams are prepared to fully transition their downstream consumption applications.

The business and technical implications of this release extend across the entire cloud analytics ecosystem. By leveraging row lineage metadata, data engineering teams can construct highly efficient incremental processing pipelines. Rather than executing full-table scans to identify modifications, downstream ETL jobs can query the sequence number directly:

SELECT *, _row_id, _last_updated_sequence_number
FROM my_catalog.namespace.clickstream
WHERE _last_updated_sequence_number > 42

This capability allows data pipelines to checkpoint processing states, ingesting only newly changed records on subsequent execution runs, thereby conserving computational resources and reducing processing durations across enterprise data estates.

AWS supports Apache Iceberg V3 across every layer of its analytics stack, providing native interoperability across storage, cataloging, ingestion, and query execution layers. Organizations can store and optimize V3 tables within Amazon S3 Tables, process and write data using Amazon EMR Spark, govern assets via AWS Glue, and execute high-performance queries using Amazon Redshift. Furthermore, both Amazon S3 Tables and the AWS Glue Data Catalog support the Iceberg REST Catalog (IRC) API, facilitating multi-engine interoperability and breaking down data silos for organizations utilizing hybrid or multi-cloud analytics strategies.

Industry observers and data architecture specialists have praised the integration, noting that native support for open standards reduces vendor lock-in while empowering enterprises to extract deeper insights from complex, multi-structured data streams. As data volumes continue to accelerate exponentially, the ability to manage petabyte-scale repositories efficiently without incurring exorbitant storage penalties or query latencies remains a paramount priority for enterprise Chief Technology Officers and data leaders.

The availability of Amazon S3 Tables support for all Apache Iceberg V3 data types spans all AWS Regions where S3 Tables are currently offered. AWS has confirmed that Apache Iceberg V3 support incurs no additional proprietary software licensing fees, with standard Amazon S3 Tables storage and request pricing remaining applicable. Organizations seeking to begin their migration or explore architectural documentation can access the official Amazon S3 Tables user guides, utilize the AWS Management Console to provision table buckets, or leverage AI-assisted tools such as the AWS MCP Server and associated plugins for technical troubleshooting and API reference searches.