AWS announced the general availability of AWS Glue 6.0, featuring 30% lower pricing, full Apache Iceberg v3 support, an upgraded Apache Spark 4.1 runtime, and real-time streaming capabilities.

Key Details

• Delivers 30% lower pricing compared to previous AWS Glue versions.
• Provides full support for Apache Iceberg v3 features, including VARIANT data type shredding, geometry/geography data types, nanosecond-precision timestamps, and unknown type handling.
• Upgrades runtime engine to Apache Spark 4.1, Python 3.12, and Scala 2.13 for faster performance.
• Introduces Spark declarative pipelines for simplified ETL authoring without manual orchestration overhead.
• Adds Arrow-native execution for Python UDFs and UDTFs, removing JVM serialization overhead and improving PySpark performance.
• Features a stateless real-time streaming mode achieving single-digit millisecond latency.
• Generally available in all AWS Regions where AWS Glue operates with no required API changes.

In This Story