Source
IQ
Toggle theme
Source
IQ
Latest
Explore
Toggle theme
Latest
Explore
Profile
/
Technologies
/
Apache Spark
Technology Profile
Apache Spark
Data Processing
Added 10d ago
Details
An open-source analytics engine for data processing.
Statistics
Default Spark shuffle partitions
200 partitions
Recommended partition data size for shuffle efficiency
100MB to 200MB per partition
Timeline
No events found for this technology.
Related Entities
Managed Service for Apache Spark
uses
Apache Software Foundation
owns
Managed Service for Apache Spark
built with
Lakehouse runtime catalog
uses
AWS Glue
built with
Sources
Aug 21, 2026
•
9d ago
AWS Glue 6.0 now available with 30% lower price and full Apache Iceberg v3 support
Source:
aws.amazon.com
Aug 19, 2026
•
11d ago
Serverless Apache Spark on Google Cloud: Architecture Choices & AI Troubleshooting
Source:
cloud.google.com
Aug 19, 2026
•
11d ago
How to modernize Apache Hive using Google Cloud’s Lakehouse runtime catalog
Source:
cloud.google.com
Aug 10, 2026
•
20d ago
How WPP operationalizes platform and data engineering for AI marketing
Source:
cloud.google.com
Jul 29, 2026
•
32d ago
The borderless Lakehouse: Bring AWS, Databricks and Snowflake data to your AI agents
Source:
cloud.google.com
SourceIQSourceIQSourceIQ